[jira] [Commented] (FLINK-12501) AvroTypeSerializer does not work with types generated by avrohugger
[ https://issues.apache.org/jira/browse/FLINK-12501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925582#comment-16925582 ] Debasish Ghosh commented on FLINK-12501: That would be very helpful Kurt > AvroTypeSerializer does not work with types generated by avrohugger > --- > > Key: FLINK-12501 > URL: https://issues.apache.org/jira/browse/FLINK-12501 > Project: Flink > Issue Type: Bug > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) >Reporter: Aljoscha Krettek >Assignee: Aljoscha Krettek >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The main problem is that the code in {{SpecificData.createSchema()}} tries to > reflectively read the {{SCHEMA$}} field, that is normally there in Avro > generated classes. However, avrohugger generates this field in a companion > object, which the reflective Java code will therefore not find. > This is also described in these ML threads: > * > [https://lists.apache.org/thread.html/5db58c7d15e4e9aaa515f935be3b342fe036e97d32e1fb0f0d1797ee@%3Cuser.flink.apache.org%3E] > * > [https://lists.apache.org/thread.html/cf1c5b8fa7f095739438807de9f2497e04ffe55237c5dea83355112d@%3Cuser.flink.apache.org%3E] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (FLINK-14012) Failed to start job for consuming Secure Kafka after the job cancel
Daebeom Lee created FLINK-14012: --- Summary: Failed to start job for consuming Secure Kafka after the job cancel Key: FLINK-14012 URL: https://issues.apache.org/jira/browse/FLINK-14012 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.9.0 Environment: * Kubernetes 1.13.2 * Flink 1.9.0 * Kafka client libary 2.2.0 Reporter: Daebeom Lee Hello, this is Daebeom Lee. h2. Background I installed Flink 1.9.0 at this our Kubernetes cluster. We use Flink session cluster. - build fatJar file and upload it at the UI, run serval jobs. At first, our jobs are good to start. But, when we cancel some jobs, the job failed This is the error code. {code:java} // code placeholder java.lang.NoClassDefFoundError: org/apache/kafka/common/security/scram/internals/ScramSaslClient at org.apache.kafka.common.security.scram.internals.ScramSaslClient$ScramSaslClientFactory.createSaslClient(ScramSaslClient.java:235) at javax.security.sasl.Sasl.createSaslClient(Sasl.java:384) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslClient$0(SaslClientAuthenticator.java:180) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:176) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.(SaslClientAuthenticator.java:168) at org.apache.kafka.common.network.SaslChannelBuilder.buildClientAuthenticator(SaslChannelBuilder.java:254) at org.apache.kafka.common.network.SaslChannelBuilder.lambda$buildChannel$1(SaslChannelBuilder.java:202) at org.apache.kafka.common.network.KafkaChannel.(KafkaChannel.java:140) at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:210) at org.apache.kafka.common.network.Selector.buildAndAttachKafkaChannel(Selector.java:334) at org.apache.kafka.common.network.Selector.registerChannel(Selector.java:325) at org.apache.kafka.common.network.Selector.connect(Selector.java:257) at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:920) at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:287) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.trySend(ConsumerNetworkClient.java:474) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:255) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215) at org.apache.kafka.clients.consumer.internals.Fetcher.getTopicMetadata(Fetcher.java:292) at org.apache.kafka.clients.consumer.KafkaConsumer.partitionsFor(KafkaConsumer.java:1803) at org.apache.kafka.clients.consumer.KafkaConsumer.partitionsFor(KafkaConsumer.java:1771) at org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.getAllPartitionsForTopics(KafkaPartitionDiscoverer.java:77) at org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.discoverPartitions(AbstractPartitionDiscoverer.java:131) at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:508) at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:529) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:393) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) at java.lang.Thread.run(Thread.java:748) {code} h2. Our workaround * I think that this is Flink JVM classloader issue. * Classloader unloads when job cancels by the way kafka client library is included fatJar. * So, I located Kafka client library to /opt/flink/lib ** /opt/flink/lib/kafka-clients-2.2.0.jar * And then all issue solved. * But there are weird points ** When Flink 1.8.1 has no problem before 2 weeks ** Before 1 week I rollback from 1.9.0 to 1.8.1, same errors occurred. ** Maybe docker image is changed at docker repository ( [https://github.com/docker-flink/docker-flink ) |https://github.com/docker-flink/docker-flink] h2. Suggestion * I'd like to know why this error occurred exactly reason after upgrade 1.9.0. * Does anybody know a better solution in this case? Thank you in advance. -- Thi
[jira] [Issue Comment Deleted] (FLINK-13975) Translate "Upcoming Events" on Chinese index.html
[ https://issues.apache.org/jira/browse/FLINK-13975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] catkint updated FLINK-13975: Comment: was deleted (was: Hi, I'm new, I’d like to do this issue, would you please assign it to me ? thx.) > Translate "Upcoming Events" on Chinese index.html > - > > Key: FLINK-13975 > URL: https://issues.apache.org/jira/browse/FLINK-13975 > Project: Flink > Issue Type: Task > Components: chinese-translation, Project Website >Reporter: Fabian Hueske >Priority: Minor > > We recently added a section for "Upcoming Events" to the index page of the > Flink website. > We need to translate "Upcoming Events" on the Chinese version of the main > page. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13025) Elasticsearch 7.x support
[ https://issues.apache.org/jira/browse/FLINK-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925580#comment-16925580 ] Stephan Ewen commented on FLINK-13025: -- Having an ElasticSearch 7 connector makes sense. I am also in favor of not trying to hack compatibility though some changes in the ES 6.x connector, as that has problematic maintenance implications (as we experienced in Kafka). So +1 from my side to create a new connector. > Elasticsearch 7.x support > - > > Key: FLINK-13025 > URL: https://issues.apache.org/jira/browse/FLINK-13025 > Project: Flink > Issue Type: New Feature > Components: Connectors / ElasticSearch >Affects Versions: 1.8.0 >Reporter: Keegan Standifer >Priority: Major > > Elasticsearch 7.0.0 was released in April of 2019: > [https://www.elastic.co/blog/elasticsearch-7-0-0-released] > The latest elasticsearch connector is > [flink-connector-elasticsearch6|https://github.com/apache/flink/tree/master/flink-connectors/flink-connector-elasticsearch6] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526487220 ## CI report: * 3c6c3d69983136fb8bfa11b28a7ba783dfe61e52 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125188905) * a9d65d8fabd938d47abd801a4686cb185f2fcf68 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125206763) * 1bbbce7f1f81598cb5a454f0e4d25adae6ed75b4 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/125425831) * d68aa63cea98320f14ff9bbe483776c8873a77ca : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125431268) * 097e1ec9e9e64ad6e9e1de16bc2937acb84ab42e : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/125726843) * 9bc0b14a50cf03c23eb4203aa2637130572047f9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125739773) * ee6669575eb9481fd7181c3a9be52e615f613e8f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125755445) * 712fdb428e78fc0d47262c6c755429a37a6be3eb : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125860362) * 16bac68a7fd82876d0ad2d56c8e7c199dc816d04 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125884245) * aeb427b0c486080e894d9753b9cfe1e385bdbd95 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126422558) * 589b8dcc5219d83ecbbbf0d79a511e440bb36e9f : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126424947) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13975) Translate "Upcoming Events" on Chinese index.html
[ https://issues.apache.org/jira/browse/FLINK-13975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925573#comment-16925573 ] catkint commented on FLINK-13975: - Hi, I'm new, I’d like to do this issue, would you please assign it to me ? thx. > Translate "Upcoming Events" on Chinese index.html > - > > Key: FLINK-13975 > URL: https://issues.apache.org/jira/browse/FLINK-13975 > Project: Flink > Issue Type: Task > Components: chinese-translation, Project Website >Reporter: Fabian Hueske >Priority: Minor > > We recently added a section for "Upcoming Events" to the index page of the > Flink website. > We need to translate "Upcoming Events" on the Chinese version of the main > page. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526484775 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 589b8dcc5219d83ecbbbf0d79a511e440bb36e9f (Mon Sep 09 10:22:54 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-12958).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] 1u0 commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
1u0 commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322167109 ## File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java ## @@ -428,6 +429,10 @@ public final void invoke() throws Exception { // See FLINK-7430 isRunning = false; } + MailboxExecutor mainMailboxExecutor = mailboxProcessor.getMainMailboxExecutor(); Review comment: There is an existing `AsyncWaitOperator` test that uses timers and blocks until it's elements are not resolved (by timeout). Without this change, the test would fail. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526487220 ## CI report: * 3c6c3d69983136fb8bfa11b28a7ba783dfe61e52 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125188905) * a9d65d8fabd938d47abd801a4686cb185f2fcf68 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125206763) * 1bbbce7f1f81598cb5a454f0e4d25adae6ed75b4 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/125425831) * d68aa63cea98320f14ff9bbe483776c8873a77ca : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125431268) * 097e1ec9e9e64ad6e9e1de16bc2937acb84ab42e : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/125726843) * 9bc0b14a50cf03c23eb4203aa2637130572047f9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125739773) * ee6669575eb9481fd7181c3a9be52e615f613e8f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125755445) * 712fdb428e78fc0d47262c6c755429a37a6be3eb : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125860362) * 16bac68a7fd82876d0ad2d56c8e7c199dc816d04 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125884245) * aeb427b0c486080e894d9753b9cfe1e385bdbd95 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126422558) * 589b8dcc5219d83ecbbbf0d79a511e440bb36e9f : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526484775 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 589b8dcc5219d83ecbbbf0d79a511e440bb36e9f (Mon Sep 09 10:08:38 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-12958).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] 1u0 commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
1u0 commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322161938 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/tasks/AbstractInvokable.java ## @@ -261,10 +261,9 @@ public void abortCheckpointOnBarrier(long checkpointId, Throwable cause) throws * Invoked when a checkpoint has been completed, i.e., when the checkpoint coordinator has received Review comment: I've added a commit message. > Maybe it should be squashed with the following commit? I prefer to separate grunt work against commits that actually introduce new behavior/feature. That commit is considered as grunt work (and also the reason why I had difficulties to find it better commit message.) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-12501) AvroTypeSerializer does not work with types generated by avrohugger
[ https://issues.apache.org/jira/browse/FLINK-12501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925549#comment-16925549 ] Kurt Young commented on FLINK-12501: Should we also fix it in 1.9 branch? > AvroTypeSerializer does not work with types generated by avrohugger > --- > > Key: FLINK-12501 > URL: https://issues.apache.org/jira/browse/FLINK-12501 > Project: Flink > Issue Type: Bug > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) >Reporter: Aljoscha Krettek >Assignee: Aljoscha Krettek >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The main problem is that the code in {{SpecificData.createSchema()}} tries to > reflectively read the {{SCHEMA$}} field, that is normally there in Avro > generated classes. However, avrohugger generates this field in a companion > object, which the reflective Java code will therefore not find. > This is also described in these ML threads: > * > [https://lists.apache.org/thread.html/5db58c7d15e4e9aaa515f935be3b342fe036e97d32e1fb0f0d1797ee@%3Cuser.flink.apache.org%3E] > * > [https://lists.apache.org/thread.html/cf1c5b8fa7f095739438807de9f2497e04ffe55237c5dea83355112d@%3Cuser.flink.apache.org%3E] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526487220 ## CI report: * 3c6c3d69983136fb8bfa11b28a7ba783dfe61e52 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125188905) * a9d65d8fabd938d47abd801a4686cb185f2fcf68 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125206763) * 1bbbce7f1f81598cb5a454f0e4d25adae6ed75b4 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/125425831) * d68aa63cea98320f14ff9bbe483776c8873a77ca : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125431268) * 097e1ec9e9e64ad6e9e1de16bc2937acb84ab42e : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/125726843) * 9bc0b14a50cf03c23eb4203aa2637130572047f9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125739773) * ee6669575eb9481fd7181c3a9be52e615f613e8f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125755445) * 712fdb428e78fc0d47262c6c755429a37a6be3eb : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125860362) * 16bac68a7fd82876d0ad2d56c8e7c199dc816d04 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125884245) * aeb427b0c486080e894d9753b9cfe1e385bdbd95 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13677) Translate "Monitoring Back Pressure" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925538#comment-16925538 ] fanrui commented on FLINK-13677: Hi, [~jark] ,can you review this pr? thanks > Translate "Monitoring Back Pressure" page into Chinese > -- > > Key: FLINK-13677 > URL: https://issues.apache.org/jira/browse/FLINK-13677 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Documentation >Affects Versions: 1.10.0 >Reporter: fanrui >Assignee: fanrui >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The page url is > [https://ci.apache.org/projects/flink/flink-docs-master/zh/monitoring/back_pressure.html] > The markdown file is located in "docs/monitoring/back_pressure.zh.md" -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9606: [FLINK-13677][docs-zh] Translate "Monitoring Back Pressure" page into Chinese
flinkbot edited a comment on issue #9606: [FLINK-13677][docs-zh] Translate "Monitoring Back Pressure" page into Chinese URL: https://github.com/apache/flink/pull/9606#issuecomment-527738237 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 17b141e4c01648d012fa722cc927803785e4204b (Mon Sep 09 09:56:23 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] 1996fanrui commented on issue #9606: [FLINK-13677][docs-zh] Translate "Monitoring Back Pressure" page into Chinese
1996fanrui commented on issue #9606: [FLINK-13677][docs-zh] Translate "Monitoring Back Pressure" page into Chinese URL: https://github.com/apache/flink/pull/9606#issuecomment-529392989 @wuchong Hi, can you review this pr? thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529346361 ## CI report: * 6def0e035c5e7f442e45a00f1f7628726aef42eb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126406386) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-10837) Kafka 2.0 test KafkaITCase.testOneToOneSources dead locked on deleteTestTopic?
[ https://issues.apache.org/jira/browse/FLINK-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925526#comment-16925526 ] Yun Tang commented on FLINK-10837: -- I come across this problem in instance [https://api.travis-ci.com/v3/job/230376914/log.txt] although failed at KafkaITCase.testTimestamps but stuck at KafkaTestBase.deleteTestTopic. > Kafka 2.0 test KafkaITCase.testOneToOneSources dead locked on deleteTestTopic? > -- > > Key: FLINK-10837 > URL: https://issues.apache.org/jira/browse/FLINK-10837 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.7.0, 1.8.0, 1.9.0 >Reporter: Piotr Nowojski >Priority: Major > Fix For: 1.7.0 > > > https://api.travis-ci.org/v3/job/452439034/log.txt > {noformat} > Tests in error: > > KafkaITCase.testOneToOneSources:97->KafkaConsumerTestBase.runOneToOneExactlyOnceTest:875->KafkaTestBase.deleteTestTopic:206->Object.wait:502->Object.wait:-2 > » TestTimedOut > {noformat} > {noformat} > java.lang.InterruptedException > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:502) > at > org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:92) > at > org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:262) > at > org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.deleteTestTopic(KafkaTestEnvironmentImpl.java:158) > at > org.apache.flink.streaming.connectors.kafka.KafkaTestBase.deleteTestTopic(KafkaTestBase.java:206) > at > org.apache.flink.streaming.connectors.kafka.KafkaConsumerTestBase.runOneToOneExactlyOnceTest(KafkaConsumerTestBase.java:875) > at > org.apache.flink.streaming.connectors.kafka.KafkaITCase.testOneToOneSources(KafkaITCase.java:97) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) > at > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.lang.Thread.run(Thread.java:748) > {noformat} -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527447402 ## CI report: * 66ade45fec93a56dde7609dffa0ad9dd241d0713 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125596416) * 51c44ece086b9d8fb8f420f783b3cf9af404f3b6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126417487) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese
flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese URL: https://github.com/apache/flink/pull/9649#issuecomment-529370466 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit fb2dd86ea8f17d546f026f779625813b1dd79e9a (Mon Sep 09 09:35:03 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] gaofeilong198810 commented on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese
gaofeilong198810 commented on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese URL: https://github.com/apache/flink/pull/9649#issuecomment-529383906 I've add some description, please have a check, thx This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13958) Job class loader may not be reused after batch job recovery
[ https://issues.apache.org/jira/browse/FLINK-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925516#comment-16925516 ] Stephan Ewen commented on FLINK-13958: -- We had a similar problem initially with RocksDB. The strange thing is that the JVM can load a library once under a given file name only, but multiple times under different file names. So in RocksDB, we rename the library file to something random and then it can be linked multiple times. > Job class loader may not be reused after batch job recovery > --- > > Key: FLINK-13958 > URL: https://issues.apache.org/jira/browse/FLINK-13958 > Project: Flink > Issue Type: Bug > Components: Runtime / Task >Affects Versions: 1.9.0 >Reporter: David Moravek >Priority: Major > > [https://lists.apache.org/thread.html/e241be9a1a10810a1203786dff3b7386d265fbe8702815a77bad42eb@%3Cdev.flink.apache.org%3E|http://example.com] > 1) We have a per-job flink cluster > 2) We use BATCH execution mode + region failover strategy > Point 1) should imply single user code class loader per task manager (because > there is only single pipeline, that reuses class loader cached in > BlobLibraryCacheManager). We need this property, because we have UDFs that > access C libraries using JNI (I think this may be fairly common use-case when > dealing with legacy code). [JDK > internals|https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/ClassLoader.java#L2466] > make sure that single library can be only loaded by a single class loader > per JVM. > When region recovery is triggered, vertices that need recover are first reset > back to CREATED stated and then rescheduled. In case all tasks in a task > manager are reset, this results in [cached class loader being > released|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/BlobLibraryCacheManager.java#L338]. > This unfortunately causes job failure, because we try to reload a native > library in a newly created class loader. > I believe the correct approach would be not to release cached class loader if > the job is recovering, even though there are no tasks currently registered > with TM. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526484775 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 16bac68a7fd82876d0ad2d56c8e7c199dc816d04 (Mon Sep 09 09:29:58 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-12958).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] 1u0 commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
1u0 commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322145045 ## File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/async/AsyncWaitOperatorFactory.java ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + *http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.streaming.api.operators.async; + +import org.apache.flink.streaming.api.datastream.AsyncDataStream; +import org.apache.flink.streaming.api.functions.async.AsyncFunction; +import org.apache.flink.streaming.api.graph.StreamConfig; +import org.apache.flink.streaming.api.operators.ChainingStrategy; +import org.apache.flink.streaming.api.operators.OneInputStreamOperatorFactory; +import org.apache.flink.streaming.api.operators.Output; +import org.apache.flink.streaming.api.operators.StreamOperator; +import org.apache.flink.streaming.api.operators.YieldingOperatorFactory; +import org.apache.flink.streaming.runtime.tasks.StreamTask; +import org.apache.flink.streaming.runtime.tasks.mailbox.execution.MailboxExecutor; + +/** + * The factory of {@link AsyncWaitOperator}. + * + * @param The output type of the operator + */ +public class AsyncWaitOperatorFactory implements OneInputStreamOperatorFactory, YieldingOperatorFactory { + private final AsyncFunction asyncFunction; + private final long timeout; + private final int capacity; + private final AsyncDataStream.OutputMode outputMode; + private MailboxExecutor mailboxExecutor; + private ChainingStrategy strategy = ChainingStrategy.HEAD; + + public AsyncWaitOperatorFactory( + AsyncFunction asyncFunction, + long timeout, + int capacity, + AsyncDataStream.OutputMode outputMode) { + this.asyncFunction = asyncFunction; + this.timeout = timeout; + this.capacity = capacity; + this.outputMode = outputMode; + } + + @Override public void setMailboxExecutor(MailboxExecutor mailboxExecutor) { + this.mailboxExecutor = mailboxExecutor; + } + + @Override public StreamOperator createStreamOperator(StreamTask containingTask, StreamConfig config, + Output output) { + AsyncWaitOperator asyncWaitOperator = new AsyncWaitOperator( + asyncFunction, + timeout, + capacity, + outputMode, + mailboxExecutor); + asyncWaitOperator.setup(containingTask, config, output); Review comment: The created ticket: [FLINK-14011](https://issues.apache.org/jira/browse/FLINK-14011). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page
flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page URL: https://github.com/apache/flink/pull/9649#issuecomment-529372783 ## CI report: * fb2dd86ea8f17d546f026f779625813b1dd79e9a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126415066) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527447402 ## CI report: * 66ade45fec93a56dde7609dffa0ad9dd241d0713 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125596416) * 51c44ece086b9d8fb8f420f783b3cf9af404f3b6 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14011) Make some fields final and initialize them during construction in AsyncWaitOperator
[ https://issues.apache.org/jira/browse/FLINK-14011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925509#comment-16925509 ] Alex commented on FLINK-14011: -- CC [~pnowojski]. > Make some fields final and initialize them during construction in > AsyncWaitOperator > --- > > Key: FLINK-14011 > URL: https://issues.apache.org/jira/browse/FLINK-14011 > Project: Flink > Issue Type: Improvement > Components: Runtime / Task >Reporter: Alex >Priority: Minor > > This is a small follow up ticket after the FLINK-12958. > With the changes introduced there, the {{AsyncWaitOperator}} is created by > {{AsyncWaitOperatorFactory}}, so some fields that initialized in the > {{setup}} method can be setup in the constructor instead. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (FLINK-14011) Make some fields final and initialize them during construction in AsyncWaitOperator
Alex created FLINK-14011: Summary: Make some fields final and initialize them during construction in AsyncWaitOperator Key: FLINK-14011 URL: https://issues.apache.org/jira/browse/FLINK-14011 Project: Flink Issue Type: Improvement Components: Runtime / Task Reporter: Alex This is a small follow up ticket after the FLINK-12958. With the changes introduced there, the {{AsyncWaitOperator}} is created by {{AsyncWaitOperatorFactory}}, so some fields that initialized in the {{setup}} method can be setup in the constructor instead. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527441976 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 51c44ece086b9d8fb8f420f783b3cf9af404f3b6 (Mon Sep 09 09:10:35 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @aljoscha [PMC] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[ https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925457#comment-16925457 ] Zhu Zhu edited comment on FLINK-13953 at 9/9/19 9:08 AM: - Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would break if we'd like to upgrade it. 3. I also found a lib [JUnitParams|https://github.com/Pragmatists/JUnitParams] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate and handle scheduler parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests maybe we can test the default scheduler("legacy" currently and "ng" later) only. And other schedulers can be verified with other scheduled travis tests or in manual ways. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? was (Author: zhuzh): Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would break if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate and handle scheduler parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests maybe we can test the default scheduler("legacy" currently and "ng" later) only. And other schedulers can be verified with other scheduled travis tests or in manual ways. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? > Facilitate enabling new Scheduler in MiniCluster Tests > -- > > Key: FLINK-13953 > URL: https://issues.apache.org/jira/browse/FLINK-13953 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination, Tests >Reporter: Gary Yao >Assignee: Zhu Zhu >Priority: Major > > Currently, tests using the {{MiniCluster}} use the legacy scheduler by > default. Once the new scheduler is implemented, we should run tests
[GitHub] [flink] flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page
flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page URL: https://github.com/apache/flink/pull/9649#issuecomment-529372783 ## CI report: * fb2dd86ea8f17d546f026f779625813b1dd79e9a : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 09:04:28 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[ https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925457#comment-16925457 ] Zhu Zhu edited comment on FLINK-13953 at 9/9/19 9:04 AM: - Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would break if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate and handle scheduler parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests maybe we can test the default scheduler("legacy" currently and "ng" later) only. And other schedulers can be verified with other scheduled travis tests or in manual ways. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? was (Author: zhuzh): Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests maybe we can test the default scheduler("legacy" currently and "ng" later) only. And other schedulers can be verified with other scheduled travis tests and in manual ways. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? > Facilitate enabling new Scheduler in MiniCluster Tests > -- > > Key: FLINK-13953 > URL: https://issues.apache.org/jira/browse/FLINK-13953 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination, Tests >Reporter: Gary Yao >Assignee: Zhu Zhu >Priority: Major > > Currently, tests using the {{MiniCluster}} use the legacy scheduler by > default. Once
[jira] [Commented] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925491#comment-16925491 ] gaofeilong commented on FLINK-13994: I have finished the translation and make a new PR on github. > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529370517 No @lord-i-will . You can rebase your branch based on the latest master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page
flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page URL: https://github.com/apache/flink/pull/9649#issuecomment-529370466 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit fb2dd86ea8f17d546f026f779625813b1dd79e9a (Mon Sep 09 09:02:13 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13994: --- Labels: pull-request-available (was: ) > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > Labels: pull-request-available > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 08:59:23 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] gaofeilong198810 opened a new pull request #9649: [FLINK-13994][docs]Translage 'Getting Started' page
gaofeilong198810 opened a new pull request #9649: [FLINK-13994][docs]Translage 'Getting Started' page URL: https://github.com/apache/flink/pull/9649 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0
flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0 URL: https://github.com/apache/flink/pull/9647#issuecomment-529282630 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit cab04a32f620acef83f46bc111512b748b4acce8 (Mon Sep 09 08:58:22 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lord-i-will edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
lord-i-will edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529368509 Now i have to close this PR? @wuchong This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lord-i-will commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
lord-i-will commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529368509 Now i have to close this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x
flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x URL: https://github.com/apache/flink/pull/9644#issuecomment-529009260 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 1472dbd78a06bc5897aa4922adfb66e82cd01f40 (Mon Sep 09 08:56:20 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 08:55:18 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925480#comment-16925480 ] Jark Wu commented on FLINK-13872: - Let's move the discussion to the pull request. > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] wuchong edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
wuchong edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529366699 Hi @lord-i-will , thanks for the contribution. There are some conflicts between your branch and master. Please rebase your branch to current master. ![image](https://user-images.githubusercontent.com/5378924/64517283-9560ab00-d322-11e9-90ca-8ba5bc0d3213.png) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529366699 Hi @lord-i-will , thanks for the contribution. There are some conflicts between your branch and master. Please rebase your branch to current master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527441976 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 66ade45fec93a56dde7609dffa0ad9dd241d0713 (Mon Sep 09 08:54:17 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @aljoscha [PMC] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Myasuka commented on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
Myasuka commented on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-529366170 It seems command `run travis` not taking effect @zentol . To rerun the CI, I have to close-reopen this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14009) Cron jobs broken due to verifying incorrect NOTICE-binary file
[ https://issues.apache.org/jira/browse/FLINK-14009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925479#comment-16925479 ] Jark Wu commented on FLINK-14009: - FLINK-13968 is fixed in 1.8.3, so I changed this issue to 1.8.3. > Cron jobs broken due to verifying incorrect NOTICE-binary file > -- > > Key: FLINK-14009 > URL: https://issues.apache.org/jira/browse/FLINK-14009 > Project: Flink > Issue Type: Bug > Components: Build System >Affects Versions: 1.10.0 >Reporter: Till Rohrmann >Assignee: Till Rohrmann >Priority: Blocker > Fix For: 1.9.1, 1.8.3 > > > With FLINK-13968 we introduced an automatic {{NOTICE-binary}} file check. > However, since we don't use the correct {{NOTICE-binary}} file (FLINK-14008) > for Scala 2.12 it fails currently our cron jobs. > I suggest to only enable the automatic {{NOTICE-binary}} files for Scala 2.11 > until FLINK-14008 has been fixed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527441976 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 66ade45fec93a56dde7609dffa0ad9dd241d0713 (Mon Sep 09 08:52:14 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @aljoscha [PMC] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14009) Cron jobs broken due to verifying incorrect NOTICE-binary file
[ https://issues.apache.org/jira/browse/FLINK-14009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu updated FLINK-14009: Fix Version/s: (was: 1.8.2) 1.8.3 > Cron jobs broken due to verifying incorrect NOTICE-binary file > -- > > Key: FLINK-14009 > URL: https://issues.apache.org/jira/browse/FLINK-14009 > Project: Flink > Issue Type: Bug > Components: Build System >Affects Versions: 1.10.0 >Reporter: Till Rohrmann >Assignee: Till Rohrmann >Priority: Blocker > Fix For: 1.9.1, 1.8.3 > > > With FLINK-13968 we introduced an automatic {{NOTICE-binary}} file check. > However, since we don't use the correct {{NOTICE-binary}} file (FLINK-14008) > for Scala 2.12 it fails currently our cron jobs. > I suggest to only enable the automatic {{NOTICE-binary}} files for Scala 2.11 > until FLINK-14008 has been fixed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service)
flinkbot edited a comment on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service) URL: https://github.com/apache/flink/pull/9643#issuecomment-528865483 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 0f85333e5d275aedc4560241bf9a268d887bb5e7 (Mon Sep 09 08:50:13 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Myasuka closed pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
Myasuka closed pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Myasuka opened a new pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
Myasuka opened a new pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604 ## What is the purpose of the change Drop all the content of removed `Checkpointed` interface. ## Brief change log ## Verifying this change This change is a trivial rework without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925475#comment-16925475 ] Jark Wu edited comment on FLINK-13994 at 9/9/19 8:49 AM: - Hi [~gaofeilong], 1. you don't need to translate the table (but the "title" and "nav-title" can be translated). 2. Yes. The link should be changed to Chinese version. If the link is started from "./", then nothing needs to change, e.g. "(./docker-playgrounds/flink-operations-playground.html)" If the link is started from "{{ site.baseurl }}", then you should add "/zh" after that, e.g. "({{ site.baseurl }}/dev/event_timestamps_watermarks.html)" needs to be changed to "({{ site.baseurl }}/zh/dev/event_timestamps_watermarks.html)". was (Author: jark): Hi [~gaofeilong], 1. you don't need to translate the table (but the title can be translated). 2. Yes. The link should be changed to Chinese version. If the link is started from "./", then nothing needs to change, e.g. "(./docker-playgrounds/flink-operations-playground.html)" If the link is started from "{{ site.baseurl }}", then you should add "/zh" after that, e.g. "({{ site.baseurl }}/dev/event_timestamps_watermarks.html)" needs to be changed to "({{ site.baseurl }}/zh/dev/event_timestamps_watermarks.html)". > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925475#comment-16925475 ] Jark Wu commented on FLINK-13994: - Hi [~gaofeilong], 1. you don't need to translate the table (but the title can be translated). 2. Yes. The link should be changed to Chinese version. If the link is started from "./", then nothing needs to change, e.g. "(./docker-playgrounds/flink-operations-playground.html)" If the link is started from "{{ site.baseurl }}", then you should add "/zh" after that, e.g. "({{ site.baseurl }}/dev/event_timestamps_watermarks.html)" needs to be changed to "({{ site.baseurl }}/zh/dev/event_timestamps_watermarks.html)". > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] TisonKun commented on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service)
TisonKun commented on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service) URL: https://github.com/apache/flink/pull/9643#issuecomment-529364151 travis fails on NOTICE-binary checker. I don't think it is relevant to this pull request. Instead, it looks like an instance of [FLINK-14009](https://issues.apache.org/jira/browse/FLINK-14009). @tillrohrmann is it relevant? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x
flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x URL: https://github.com/apache/flink/pull/9644#issuecomment-529009260 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 1472dbd78a06bc5897aa4922adfb66e82cd01f40 (Mon Sep 09 08:48:10 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lirui-apache commented on a change in pull request #9644: [FLINK-13931][hive] Support Hive version 2.0.x
lirui-apache commented on a change in pull request #9644: [FLINK-13931][hive] Support Hive version 2.0.x URL: https://github.com/apache/flink/pull/9644#discussion_r322126762 ## File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/TableEnvHiveConnectorTest.java ## @@ -114,6 +114,10 @@ public void testGetNonExistingFunction() throws Exception { public void testDifferentFormats() throws Exception { String[] formats = new String[]{"orc", "parquet", "sequencefile", "csv"}; for (String format : formats) { + if (format.equals("orc") && hmsClient.getHiveVersion().startsWith("2.0")) { + // Ignore orc test for Hive version 2.0.x for now due to FLINK-13998 Review comment: Do we plan to test against 2.0.1 as well? If so, we also need to skip parquet and sequencefile tests for 2.0.1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925469#comment-16925469 ] gaofeilong commented on FLINK-13994: hi [~jark], I have two question when i translating this page:[https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md:|https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md] # should I translate the table on the top of the page? # the link in this page is relative path, should I change the link in this page to zh's link? > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-14010) Dispatcher & JobManagers don't give up leadership when AM is shut down
[ https://issues.apache.org/jira/browse/FLINK-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925460#comment-16925460 ] TisonKun commented on FLINK-14010: -- CC [~StephanEwen] [~till.rohrmann] [~xiaogang.shi] Here comes a high-level problem, do we explicitly constrain Dispatcher, ResourceManager and JobManagers run on one process? 1. the usage of reference to {{JobManagerGateway}} in Dispatcher already infers that we require this. 2. back to the design of FLIP-6, we have a global singleton of Dispatcher, and for each job, launch a JobManager and ResourceManager. The implementation diverges quite a lot. Could you please provide any background? 3. if we explicitly constrain as above, we actually need not to start leader election services per components, actually, we can use the abstraction and layout as below: - start a leader election service per dispatcher-resource-manager component, in cluster entrypoint level. It will participant the election and all metadata commits are delegate to this service. - all cluster level components that need to publish their address, such as Dispatcher, ResourceManager and WebMonitor publish their address via this leader election service. - Actors can be started as {{PermanentlyFencedRpcEndpoint}} and thus we survive from handling a lot of mutable shared state among leadership epoch. Specifically, cluster entrypoint acts as DispatcherRunner and so on, like JobManagerRunner to JobMaster. See also [this branch|https://github.com/tillrohrmann/flink/commits/removeSuspendFromJobMaster]. - back to this issue, cluster entrypoint({{YARNClusterEntrypoint}} maybe) reacts to AMRM request and thus all components can be required to shutdown properly. > Dispatcher & JobManagers don't give up leadership when AM is shut down > -- > > Key: FLINK-14010 > URL: https://issues.apache.org/jira/browse/FLINK-14010 > Project: Flink > Issue Type: Bug > Components: Deployment / YARN, Runtime / Coordination >Affects Versions: 1.7.2, 1.8.1, 1.9.0, 1.10.0 >Reporter: TisonKun >Priority: Critical > > In YARN deployment scenario, YARN RM possibly launches a new AM for the job > even if the previous AM does not terminated, for example, when AMRM heartbeat > timeout. This is a common case that RM will send a shutdown request to the > previous AM and expect the AM shutdown properly. > However, currently in {{YARNResourceManager}}, we handle this request in > {{onShutdownRequest}} which simply close the {{YARNResourceManager}} *but not > Dispatcher and JobManagers*. Thus, Dispatcher and JobManager launched in new > AM cannot be granted leadership properly. Visually, > on previous AM: Dispatcher leader, JM leaders > on new AM: ResourceManager leader > since on client side or in per-job mode, JobManager address and port are > configured as the new AM, the whole cluster goes into an unrecoverable > inconsistent status: client all queries the dispatcher on new AM who is now > the leader. Briefly, Dispatcher and JobManagers on previous AM do not give up > their leadership properly. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[ https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925457#comment-16925457 ] Zhu Zhu edited comment on FLINK-13953 at 9/9/19 8:30 AM: - Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests maybe we can test the default scheduler("legacy" currently and "ng" later) only. And other schedulers can be verified with other scheduled travis tests and in manual ways. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? was (Author: zhuzh): Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests only the default scheduler("legacy" currently and "ng" later) is used. And scheduled travis builds can be added for other schedulers. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? > Facilitate enabling new Scheduler in MiniCluster Tests > -- > > Key: FLINK-13953 > URL: https://issues.apache.org/jira/browse/FLINK-13953 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination, Tests >Reporter: Gary Yao >Assignee: Zhu Zhu >Priority: Major > > Currently, tests using the {{MiniCluster}} use the legacy scheduler by > def
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[ https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925457#comment-16925457 ] Zhu Zhu edited comment on FLINK-13953 at 9/9/19 8:28 AM: - Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests only the default scheduler("legacy" currently and "ng" later) is used. And scheduled travis builds can be added for other schedulers. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? was (Author: zhuzh): Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests only the default scheduler("legacy" currently and "ng" later) is used. And scheduled travis builds can be added for other schedulers. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? > Facilitate enabling new Scheduler in MiniCluster Tests > -- > > Key: FLINK-13953 > URL: https://issues.apache.org/jira/browse/FLINK-13953 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination, Tests >Reporter: Gary Yao >Assignee: Zhu Zhu >Priority: Major > > Currently, tests using the {{MiniCluster}} use the legacy scheduler by > default. Once the new scheduler is i
[jira] [Commented] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[ https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925457#comment-16925457 ] Zhu Zhu commented on FLINK-13953: - Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests) 1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class. 2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it. 3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance. One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime. Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better. With it we can run MiniCluster tests for different schedulers with different profiles. For per-commit tests only the default scheduler("legacy" currently and "ng" later) is used. And scheduled travis builds can be added for other schedulers. A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng". What do you think? > Facilitate enabling new Scheduler in MiniCluster Tests > -- > > Key: FLINK-13953 > URL: https://issues.apache.org/jira/browse/FLINK-13953 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination, Tests >Reporter: Gary Yao >Assignee: Zhu Zhu >Priority: Major > > Currently, tests using the {{MiniCluster}} use the legacy scheduler by > default. Once the new scheduler is implemented, we should run tests with the > new scheduler enabled. However, it is not expected that all tests will pass > immediately. Therefore, it should be possible to enable the new scheduler for > a subset of tests. > *Acceptance Criteria* > * Subset of tests using {{MiniCluster}} can be run on a per-commit basis (on > Travis) against new scheduler and also legacy scheduler -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526484775 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 16bac68a7fd82876d0ad2d56c8e7c199dc816d04 (Mon Sep 09 08:20:41 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-12958).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322080057 ## File path: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/tasks/StreamTaskTest.java ## @@ -1380,6 +1398,54 @@ protected void performDefaultAction(DefaultActionContext context) throws Excepti } } + /** +* A task that checks that some StreamTask methods are called only in the main task's thread. Review comment: I would rephrase this comment and move it above `testThreadInvariants`. Otherwise it's quite difficult to understand what this test is suppose to do. Also could you copy paste this also as a commit message? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322087304 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/tasks/AbstractInvokable.java ## @@ -261,10 +261,9 @@ public void abortCheckpointOnBarrier(long checkpointId, Throwable cause) throws * Invoked when a checkpoint has been completed, i.e., when the checkpoint coordinator has received Review comment: @1u0 ? (You have resolved the comment, but haven't responded and haven't updated the commit message) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322079010 ## File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java ## @@ -428,6 +429,10 @@ public final void invoke() throws Exception { // See FLINK-7430 isRunning = false; } + MailboxExecutor mainMailboxExecutor = mailboxProcessor.getMainMailboxExecutor(); Review comment: Was this addressed @1u0? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322111247 ## File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/async/Emitter.java ## @@ -80,9 +88,14 @@ public void run() { try { while (running) { LOG.debug("Wait for next completed async stream element result."); - AsyncResult streamElementEntry = streamElementQueue.peekBlockingly(); - - output(streamElementEntry); + AsyncResult asyncResult = streamElementQueue.peekBlockingly(); + executor.submit(() -> { + try { + output(asyncResult); + } catch (InterruptedException e) { Review comment: Why do we need this `InterruptedException`? I see that it's coming (and it is the same in master branch) from `streamElementQueue.poll()`, however I don't understand it: 1. We shouldn't be blocking neither in the mailbox thread (this version) nor under the checkpoint lock (master version), so this already looks suspicious. 2. `poll` method that blocks is already "mildly" surprising contract on it's own 3. From the code it looks like this `poll` call should never block and changing it to a proper poll (returning `Optional` or `null`) and adding a check state that the element is present, would work just as well but would be better/cleaner. @AHeise I remember that you were also rising some objections about the `Emitter` code and that it could be simplified and you wanted to work on that later. Did you have this in mind as well? Am I missing something? If not can you @AHeise create a JIRA ticket for cleaning up those things? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529346361 ## CI report: * 6def0e035c5e7f442e45a00f1f7628726aef42eb : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126406386) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14010) Dispatcher & JobManagers don't give up leadership when AM is shut down
TisonKun created FLINK-14010: Summary: Dispatcher & JobManagers don't give up leadership when AM is shut down Key: FLINK-14010 URL: https://issues.apache.org/jira/browse/FLINK-14010 Project: Flink Issue Type: Bug Components: Deployment / YARN, Runtime / Coordination Affects Versions: 1.9.0, 1.8.1, 1.7.2, 1.10.0 Reporter: TisonKun In YARN deployment scenario, YARN RM possibly launches a new AM for the job even if the previous AM does not terminated, for example, when AMRM heartbeat timeout. This is a common case that RM will send a shutdown request to the previous AM and expect the AM shutdown properly. However, currently in {{YARNResourceManager}}, we handle this request in {{onShutdownRequest}} which simply close the {{YARNResourceManager}} *but not Dispatcher and JobManagers*. Thus, Dispatcher and JobManager launched in new AM cannot be granted leadership properly. Visually, on previous AM: Dispatcher leader, JM leaders on new AM: ResourceManager leader since on client side or in per-job mode, JobManager address and port are configured as the new AM, the whole cluster goes into an unrecoverable inconsistent status: client all queries the dispatcher on new AM who is now the leader. Briefly, Dispatcher and JobManagers on previous AM do not give up their leadership properly. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529346361 ## CI report: * 6def0e035c5e7f442e45a00f1f7628726aef42eb : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925443#comment-16925443 ] Lord i Will edited comment on FLINK-13872 at 9/9/19 7:59 AM: - [~jark] [~fhueske] it seems that there's a conflict. But i have rebased on the master. Don't know why~ was (Author: a64011144): [~jark] [~fhueske] it seems that there've a conflict. But i have rebased on the master. Don't know why~ > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925443#comment-16925443 ] Lord i Will commented on FLINK-13872: - [~jark] [~fhueske] it seems that there've a conflict. But i have rebased on the master. Don't know why~ > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13619) Update FlinkKafkaProducerMigrationOperatorTest to restore from 1.9 savepoint
[ https://issues.apache.org/jira/browse/FLINK-13619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925439#comment-16925439 ] vinoyang commented on FLINK-13619: -- [~pnowojski] So how to generate the binary file for this migration test? Or how to verify this migration test? > Update FlinkKafkaProducerMigrationOperatorTest to restore from 1.9 savepoint > > > Key: FLINK-13619 > URL: https://issues.apache.org/jira/browse/FLINK-13619 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Kafka, Tests >Affects Versions: 1.10.0 >Reporter: Till Rohrmann >Assignee: vinoyang >Priority: Blocker > Fix For: 1.10.0 > > > Update {{FlinkKafkaProducerMigrationOperatorTest}} to restore from 1.9 > savepoint. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 07:55:45 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13872: --- Labels: pull-request-available (was: ) > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] lord-i-will opened a new pull request #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
lord-i-will opened a new pull request #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648 Please have a check, thx. @wuchong @fhueske This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14009) Cron jobs broken due to verifying incorrect NOTICE-binary file
Till Rohrmann created FLINK-14009: - Summary: Cron jobs broken due to verifying incorrect NOTICE-binary file Key: FLINK-14009 URL: https://issues.apache.org/jira/browse/FLINK-14009 Project: Flink Issue Type: Bug Components: Build System Affects Versions: 1.10.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.8.2, 1.9.1 With FLINK-13968 we introduced an automatic {{NOTICE-binary}} file check. However, since we don't use the correct {{NOTICE-binary}} file (FLINK-14008) for Scala 2.12 it fails currently our cron jobs. I suggest to only enable the automatic {{NOTICE-binary}} files for Scala 2.11 until FLINK-14008 has been fixed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (FLINK-14008) Flink Scala 2.12 binary distribution contains incorrect NOTICE-binary file
Till Rohrmann created FLINK-14008: - Summary: Flink Scala 2.12 binary distribution contains incorrect NOTICE-binary file Key: FLINK-14008 URL: https://issues.apache.org/jira/browse/FLINK-14008 Project: Flink Issue Type: Bug Components: Build System Affects Versions: 1.9.0, 1.8.1, 1.10.0 Reporter: Till Rohrmann Fix For: 1.10.0 The Flink Scala {{2.12}} binary distribution contains an incorrect NOTICE-binary file. The problem is that we don't update the Scala version of the Scala dependencies listed in the {{NOTICE-binary}} file. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13968) Add travis check for the correctness of the binary licensing
[ https://issues.apache.org/jira/browse/FLINK-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925424#comment-16925424 ] Till Rohrmann commented on FLINK-13968: --- Failing cron job: https://api.travis-ci.org/v3/job/582421469/log.txt > Add travis check for the correctness of the binary licensing > > > Key: FLINK-13968 > URL: https://issues.apache.org/jira/browse/FLINK-13968 > Project: Flink > Issue Type: Improvement > Components: Travis >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0, 1.9.1, 1.8.3 > > Time Spent: 20m > Remaining Estimate: 0h > > Since the binary licensing must be updated manually whenever the packaging of > flink-dist or contained jars is modified we should add an automatic check to > ensure the licensing is updated appropriately. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13968) Add travis check for the correctness of the binary licensing
[ https://issues.apache.org/jira/browse/FLINK-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925423#comment-16925423 ] Till Rohrmann commented on FLINK-13968: --- I think this change breaks our cron jobs, because the NOTICE-binary files isn't correct when building with Scala 2.12. > Add travis check for the correctness of the binary licensing > > > Key: FLINK-13968 > URL: https://issues.apache.org/jira/browse/FLINK-13968 > Project: Flink > Issue Type: Improvement > Components: Travis >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0, 1.9.1, 1.8.3 > > Time Spent: 20m > Remaining Estimate: 0h > > Since the binary licensing must be updated manually whenever the packaging of > flink-dist or contained jars is modified we should add an automatic check to > ensure the licensing is updated appropriately. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9432: [FLINK-13653][sql-client] ResultStore should avoid using RowTypeInfo …
flinkbot edited a comment on issue #9432: [FLINK-13653][sql-client] ResultStore should avoid using RowTypeInfo … URL: https://github.com/apache/flink/pull/9432#issuecomment-521081788 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 9f196705610a4b48715dcff37610dee4438fc010 (Mon Sep 09 07:22:34 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @docete * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0
flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0 URL: https://github.com/apache/flink/pull/9647#issuecomment-529282630 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit cab04a32f620acef83f46bc111512b748b4acce8 (Mon Sep 09 07:07:18 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services