Hi Till, Thanks for taking a look and for opening the PR.
Previous options that specified a "pause" seem to be semantically different from new options that specify a "timeout", so probably best to remove them from code and documentation (old options are also still listed on https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html) Thomas On Wed, Jun 24, 2020 at 1:36 AM Till Rohrmann <trohrm...@apache.org> wrote: > Thanks for testing the RC and the feedback Thomas. The problem with the > taskmanager options is that the old > (taskmanager.initial-registration-pause) and new options > (cluster.registration.initial-timeout) don't have the same type. The old > options have not been used for a long time (since version 1.5.0) and we > wanted to remove them. As part of the removal, we added the old keys as > deprecated options for the new ones. I believe this was a mistake. I've > opened a PR to remove the deprecated keys from the new ConfigOptions [1]. > > Please be aware that > > "taskmanager.initial-registration-pause": "500ms", > "taskmanager.max-registration-pause": "5s", > "taskmanager.refused-registration-pause": "5s", > > Shouldn't have any effects anymore (since version 1.5.0). > > [1] https://github.com/apache/flink/pull/12763 > > Cheers, > Till > > On Wed, Jun 24, 2020 at 4:17 AM Zhijiang <wangzhijiang...@aliyun.com > .invalid> > wrote: > > > Hi Thomas, > > > > Thanks for these valuable feedbacks and suggestions, and I think they are > > very helpful for making us better. > > > > I can give an direct answer for this issue: > > > checkpoint alignment buffered metric missing - note that this job isn't > > using the new unaligned checkpointing that should be opt-in. > > > > The metric of checkpoint alignment buffered would be always 0 now, no > > matter with unaligned checkpointing or not, so we removed this metric > > directly. > > The motivation for such change is from reducing in-flight buffers to > speed > > up checkpoint somehow. The upstream side would block sending any > following > > buffers after sending the barrier until receiving the alignment > > notification from downstream side. Therefore, the downstream side never > > needs to cache > > buffers for blocked channels during alignment. We also illustrated such > > changes in release notes for attention by link [1]. > > > > [1] > > > https://github.com/apache/flink/pull/12699/files#diff-eaa874e007e88f283e96de2d61cc4140R174 > > > > Best, > > Zhijiang > > ------------------------------------------------------------------ > > From:Thomas Weise <t...@apache.org> > > Send Time:2020年6月24日(星期三) 06:51 > > To:dev <dev@flink.apache.org> > > Cc:zhijiang <zhiji...@apache.org> > > Subject:Re: [ANNOUNCE] Apache Flink 1.11.0, release candidate #2 > > > > Hi, > > > > Thanks for putting together the RC! > > > > I have some preliminary feedback from testing with commit > > 934f91ead00fd658333f65ffa37ab60bd5ffd99b > > > > An internal benchmark application that reads from Kinesis and checkpoints > > ~12GB performs comparably to 1.10.1 > > > > There were a few issues hit upgrading our codebase that may be worthwhile > > considering, please see details below. > > > > Given my observations over the past few releases, I would like to suggest > > that the community introduces a log of incompatible changes to be > published > > with the release notes. Though it is possible to analyze git history when > > hitting compile errors, there are more subtle changes that can make > > upgrades unnecessarily time-consuming. Contributors introducing such > > changes are probably in the best position to document. > > > > I'm planning to try this or the next RC with a couple more applications. > > > > Cheers, > > Thomas > > > > * notifyCheckpointAborted needed to be implemented > > for org.apache.flink.runtime.state.CheckpointListener - can we have the > > default implementation in the interface so that users aren't forced to > > change their implementations > > > > * following deprecated configuration values had to be modified to get > > the job running: > > > > "taskmanager.initial-registration-pause": "500ms", > > "taskmanager.max-registration-pause": "5s", > > "taskmanager.refused-registration-pause": "5s", > > > > The error message was: > > > > Could not parse value '500ms' for key > > 'cluster.registration.initial-timeout'.\n\tat > > > > > org.apache.flink.configuration.Configuration.getOptional(Configuration.java:753)\n\tat > > > > > org.apache.flink.configuration.Configuration.getLong(Configuration.java:298)\n\tat > > > > > org.apache.flink.runtime.registration.RetryingRegistrationConfiguration.fromConfiguration(RetryingRegistrationConfiguration.java:72)\n\tat > > > > > org.apache.flink.runtime.taskexecutor.TaskManagerServicesConfiguration.fromConfiguration(TaskManagerServicesConfiguration.java:262)\n\tat > > > > Though easy to fix, it's unfortunate that values are now treated > > differently. > > > > * checkpoint alignment buffered metric missing - note that this job isn't > > using the new unaligned checkpointing that should be opt-in. > > > > * -import org.apache.flink.table.api.java.StreamTableEnvironment; > > +import org.apache.flink.table.api.bridge.java.StreamTableEnvironment; > > > > * -ClientUtils.executeProgram(DefaultExecutorServiceLoader.INSTANCE, > > config, program.build()); > > +ClientUtils.executeProgram(DefaultExecutorServiceLoader.INSTANCE, > > config, program.build(), > > false, false); > > > > * ProcessingTimeCallback removed from StreamingFileSink > > > > > > On Wed, Jun 17, 2020 at 6:29 AM Piotr Nowojski <pnowoj...@apache.org> > > wrote: > > > > > Hi all, > > > > > > I would like to give an update about the RC2 status. We are now waiting > > for > > > a green azure build on one final bug fix before creating RC2. This bug > > fix > > > should be merged late afternoon/early evening Berlin time, so RC2 will > be > > > hopefully created tomorrow morning. Until then I would ask to not > > > merge/backport commits to release-1.11 branch, including bug fixes. If > > you > > > have something that's truly essential and should be treated as a > release > > > blocker, please reach out to me or Zhijiang. > > > > > > Best, > > > Piotr Nowojski > > > > > > > >
