Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Stephan Ewen
+1 (binding) - Verified commit history, looks good => stumbled over the changes in the "create_release_branch.sh ", which are present in each release commit. [1] => agree that these are not an issue, because this is an out-of-band release - Release notes for 1.14.2 are off,

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-15 Thread Stephan Ewen
> Hence I don't think the jars help in this case. > > On 15/12/2021 10:42, Stephan Ewen wrote: > > Given that these artifacts are published already, users can use them if > > they want to update now: > > > > For example: > > https://search.maven.org

Re: [ANNOUNCE] New Apache Flink Committer - Matthias Pohl

2021-12-15 Thread Stephan Ewen
Congratulations, Matthias, and welcome to the Flink committers! On Mon, Dec 13, 2021 at 4:39 AM 刘建刚 wrote: > Congratulations! > > Best > Liu Jiangang > > Nicholas Jiang 于2021年12月13日周一 11:23写道: > > > Congratulations, Matthias! > > > > Best, > > Nicholas Jiang > > >

Re: [ANNOUNCE] New Apache Flink Committer - Ingo Bürk

2021-12-15 Thread Stephan Ewen
Congrats and welcome! On Mon, Dec 13, 2021 at 4:39 AM 刘建刚 wrote: > Congratulations! > > Best > Liu Jiangang > > Nicholas Jiang 于2021年12月13日周一 11:28写道: > > > Congratulations, Ingo! > > > > Best, > > Nicholas Jiang > > >

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-15 Thread Stephan Ewen
Given that these artifacts are published already, users can use them if they want to update now: For example: https://search.maven.org/artifact/org.apache.flink/flink-core/1.14.1/jar Just for the users that really want to update now (rather than rely on the mitigation via config) and are not as

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-13 Thread Stephan Ewen
+1 (binding) - Verified that commit history is identical to previous release (except dependency upgrade and release version commit) - Verified that the source releases reference updated dependency and binary releases contain updated dependency - Blog post looks good - ran bundled examples

[DISCUSS] Immediate dedicated Flink releases for log4j vulnerability

2021-12-12 Thread Stephan Ewen
Hi all! Without doubt, you heard about the log4j vulnerability [1]. There is an advisory blog post on how to mitigate this in Apache Flink [2], which involves setting a config option and restarting the processes. That is fortunately a relatively simple fix. Despite this workaround, I think we

Re: [VOTE] FLIP-188 Introduce Built-in Dynamic Table Storage

2021-11-24 Thread Stephan Ewen
n nice. > > > > I also added some comments in the DISCUSS thread. Let's hope we can > > resolve those soon. > > > > Regards, > > Timo > > > > On 12.11.21 16:36, Stephan Ewen wrote: > > > Hi all! > > > > > > I

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

2021-11-23 Thread Stephan Ewen
Thanks for digging into this. Regarding this query: INSERT INTO the_table SELECT window_end, COUNT(*) FROM (TUMBLE(TABLE interactions, DESCRIPTOR(ts), INTERVAL '5' MINUTES)) GROUP BY window_end HAVING now() - window_end <= INTERVAL '14' DAYS; I am not sure I understand what the

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

2021-11-16 Thread Stephan Ewen
Hi Jingsong! Thank you for all the explanations. To follow up on the points: (1) Log implementation Good to hear you are looking to make this extensible. (2) change tracking Understood, makes sense (used for re-processing). (3) Log Scan Startup mode. Your explanation makes sense. As I

Re: [DISCUSS] Update Policy for old releases

2021-11-12 Thread Stephan Ewen
I am of a bit different opinion here, I don't think LTS is the best way to go. To my understanding, the big issue faced by users is that an upgrade is simply too hard. Too many things change in subtle ways, you cannot just take the previous application and configuration and expect it to run the

Re: [VOTE] FLIP-188 Introduce Built-in Dynamic Table Storage

2021-11-12 Thread Stephan Ewen
Hi all! I have a few questions on the design still, posted those in the [DISCUSS] thread. It would be great to clarify those first before concluding this vote. Thanks, Stephan On Fri, Nov 12, 2021 at 7:22 AM Jark Wu wrote: > +1 (binding) > > Thanks for the great work Jingsong! > > Best, >

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

2021-11-12 Thread Stephan Ewen
Hi all! Thank you for the writeup of this feature. I like the general direction a lot. There are some open questions and confusing details still, which I think we need to clarify first to make this feature really good. Below are questions/suggestions on the FLIP: Best, Stephan ===

[jira] [Created] (FLINK-24852) Cleanup of Orphaned Incremental State Artifacts

2021-11-09 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-24852: Summary: Cleanup of Orphaned Incremental State Artifacts Key: FLINK-24852 URL: https://issues.apache.org/jira/browse/FLINK-24852 Project: Flink Issue Type

Re: [DISCUSS] Creating an external connector repository

2021-10-28 Thread Stephan Ewen
Thank you all, for the nice discussion! >From my point of view, I very much like the idea of putting connectors in a separate repository. But I would argue it should be part of Apache Flink, similar to flink-statefun, flink-ml, etc. I share many of the reasons for that: - As argued many times,

Re: [NOTICE] CiBot improvements

2021-10-11 Thread Stephan Ewen
Great initiative, thanks for doing this! On Mon, Oct 11, 2021 at 10:52 AM Till Rohrmann wrote: > Thanks a lot for this effort Chesnay! The improvements sound really good. > > Cheers, > Till > > On Mon, Oct 11, 2021 at 8:46 AM David Morávek wrote: > > > Nice! Thanks for the effort Chesnay, this

[jira] [Created] (FLINK-24366) Unnecessary/misleading error message about failing restores when tasks are already canceled.

2021-09-23 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-24366: Summary: Unnecessary/misleading error message about failing restores when tasks are already canceled. Key: FLINK-24366 URL: https://issues.apache.org/jira/browse/FLINK-24366

[jira] [Created] (FLINK-24343) Revisit Scheduler and Coordinator Startup Procedure

2021-09-21 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-24343: Summary: Revisit Scheduler and Coordinator Startup Procedure Key: FLINK-24343 URL: https://issues.apache.org/jira/browse/FLINK-24343 Project: Flink Issue

Re: [VOTE] FLIP-173: Support DAG of algorithms

2021-09-17 Thread Stephan Ewen
I think this will be a useful addition. Regarding the API and specific design decisions: I think this looks ok. I didn't dig very deep and would be fine to just go with the author's proposal. The main motivation for having a separate flink-ml repository was to develop more easily, make changes

[jira] [Created] (FLINK-24255) Test Environment / Mini Cluster do not forward configuration.

2021-09-10 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-24255: Summary: Test Environment / Mini Cluster do not forward configuration. Key: FLINK-24255 URL: https://issues.apache.org/jira/browse/FLINK-24255 Project: Flink

Re: [VOTE] Apache Flink Stateful Functions 3.1.0, release candidate #1

2021-08-26 Thread Stephan Ewen
+1 (binding) - Build project on Java 8 (maven command line) with full end to end tests - Compiled and ran tests in IDE (IntelliJ) with Java 11 (possible after some manual config, see comment below to automate this) - No binaries in the distribution - Verified license and notice files -

[ANNOUNCE] Dropping "CheckpointConfig.setPreferCheckpointForRecovery()"

2021-08-24 Thread Stephan Ewen
Hi Flink Community! A quick heads-up: We suggest removing the setting "CheckpointConfig.setPreferCheckpointForRecovery()" [1]. The setting has been deprecated since Flink 1.12 and is strongly discouraged, because it can lead to data loss or data duplication in different scenarios. Please see

[jira] [Created] (FLINK-23843) Exceptions during "SplitEnumeratorContext.runInCoordinatorThread()" should cause Global Failure instead of Process Kill

2021-08-17 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23843: Summary: Exceptions during "SplitEnumeratorContext.runInCoordinatorThread()" should cause Global Failure instead of Process Kill Key: FLINK-23843 URL: https://issues.

[jira] [Created] (FLINK-23842) Add log messages for reader registrations and split requests.

2021-08-17 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23842: Summary: Add log messages for reader registrations and split requests. Key: FLINK-23842 URL: https://issues.apache.org/jira/browse/FLINK-23842 Project: Flink

Re: [VOTE] FLIP-180: Adjust StreamStatus and Idleness definition

2021-08-09 Thread Stephan Ewen
+1 (binding) On Mon, Aug 9, 2021 at 12:08 PM Till Rohrmann wrote: > +1 (binding) > > Cheers, > Till > > On Thu, Aug 5, 2021 at 9:09 PM Arvid Heise wrote: > > > Dear devs, > > > > I'd like to open a vote on FLIP-180: Adjust StreamStatus and Idleness > > definition [1] which was discussed in

[jira] [Created] (FLINK-23649) Add RocksDB packages to parent-first classloading patterns.

2021-08-05 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23649: Summary: Add RocksDB packages to parent-first classloading patterns. Key: FLINK-23649 URL: https://issues.apache.org/jira/browse/FLINK-23649 Project: Flink

[jira] [Created] (FLINK-23630) Make EventTimeWindowCheckpointingITCase and LocalRecoveryITCase run on Windows.

2021-08-04 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23630: Summary: Make EventTimeWindowCheckpointingITCase and LocalRecoveryITCase run on Windows. Key: FLINK-23630 URL: https://issues.apache.org/jira/browse/FLINK-23630

[jira] [Created] (FLINK-23629) Remove redundant test cases in EventTimeWindowCheckpointingITCase

2021-08-04 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23629: Summary: Remove redundant test cases in EventTimeWindowCheckpointingITCase Key: FLINK-23629 URL: https://issues.apache.org/jira/browse/FLINK-23629 Project: Flink

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Stephan Ewen
graded > from Flink 1.9 to 1.13. > > Do we know why there's such a huge performance regression? Can we improve > this somehow with some flag tweaking? It would be great if we see a more in > depth explanation of the gains vs losses of upgrading. > > On Wed, Aug 4, 2021 at 3:08 PM S

[ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Stephan Ewen
Hi all! *!!! If you are a big user of the Embedded RocksDB State Backend and have performance sensitive workloads, please read this !!!* I want to quickly raise some awareness for a RocksDB version upgrade we plan to do, and some possible impact on application performance. *We plan to upgrade

Re: [VOTE] FLIP-150: Introduce Hybrid Source

2021-07-28 Thread Stephan Ewen
+1 (binding) Would be good to move this across the finish line. It seems this thread is held up mainly by formalities at this point. On Fri, Jul 2, 2021 at 4:27 AM Israel Ekpo wrote: > +1 (non-binding) > > On Thu, Jul 1, 2021 at 6:45 PM Elkhan Dadashov > wrote: > > > +1 (non-binding) > > > >

Re: [DISCUSS] Address deprecation warnings when upgrading dependencies

2021-07-15 Thread Stephan Ewen
s on the table on how to actually do the > migration; we can use hamcrest of course, or create a small wrapper in > our test utils that retains the signature junit signature (then we'd > just have to adjust imports). > > On 14/07/2021 16:33, Stephan Ewen wrote: > > @Chesnay - c

Re: [DISCUSS] Address deprecation warnings when upgrading dependencies

2021-07-14 Thread Stephan Ewen
@Chesnay - can you elaborate on this? In the classes I worked with so far, it was a 1:1 replacement of "org.junit.Assert.assertThat()" to "org.hamcrest.MatcherAssert.assertThat()". What other migration should happen there? On Wed, Jul 14, 2021 at 11:13 AM Chesnay Schepler wrote: > It may be

Re: [DISCUSS] Address deprecation warnings when upgrading dependencies

2021-07-14 Thread Stephan Ewen
never touching a test class, replacing Junit's > >> assertThat with Hamcrest's version which felt quite tedious. > >> > >> Cheers, > >> Till > >> > >> On Tue, Jul 13, 2021 at 6:15 PM Stephan Ewen wrote: > >> > >>> Hi all! > >>

Re: [DISCUSS] FLIP-182: Watermark alignment

2021-07-13 Thread Stephan Ewen
@Eron Wright The per-split watermarks are the default in the new source interface (FLIP-27) and come for free if you use the SplitReader. Based on that, it is also possible to unsubscribe individual splits to solve the alignment in the case where operators have multiple splits assigned. Piotr

[DISCUSS] Address deprecation warnings when upgrading dependencies

2021-07-13 Thread Stephan Ewen
Hi all! I would like to propose that we make it a project standard that when upgrading a dependency, deprecation issues arising from that need to be fixed in the same step. If the new dependency version deprecates a method in favor of another method, all usages in the code need to be replaced

Re: [DISCUSS]FLIP-170 Adding Checkpoint Rejection Mechanism

2021-07-13 Thread Stephan Ewen
er might be more acceptable. > > > [1] > https://github.com/debezium/debezium-design-documents/blob/main/DDD-3.md > [2] > > https://github.com/ververica/flink-cdc-connectors/commit/c6ca6c187471b538a9774258d2572194e1bb855b > > Stephan Ewen 于2021年7月13日周二 上午1:25写道: > >

Re: [DISCUSS]FLIP-170 Adding Checkpoint Rejection Mechanism

2021-07-12 Thread Stephan Ewen
Hi! Thanks for writing this FLIP, and interesting idea. I would like to understand a bit better why exactly we need this, and what our alternative options are. My main concerns are the following: *(1) Can we achieve this without changing the checkpointing mechanism?* The checkpoint mechanism

[jira] [Created] (FLINK-23301) StateFun HTTP Ingress

2021-07-07 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23301: Summary: StateFun HTTP Ingress Key: FLINK-23301 URL: https://issues.apache.org/jira/browse/FLINK-23301 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23281) StateFun - Simplify Getting Stared Experience with HTTP Ingresses and Egresses

2021-07-06 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23281: Summary: StateFun - Simplify Getting Stared Experience with HTTP Ingresses and Egresses Key: FLINK-23281 URL: https://issues.apache.org/jira/browse/FLINK-23281

[jira] [Created] (FLINK-23261) StateFun - HTTP State Access Interface

2021-07-05 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23261: Summary: StateFun - HTTP State Access Interface Key: FLINK-23261 URL: https://issues.apache.org/jira/browse/FLINK-23261 Project: Flink Issue Type

Re: [DISCUSS] Feedback Collection Jira Bot

2021-07-01 Thread Stephan Ewen
“Does this still happen on Flink 1.x?” > > > - > > > > > > “Are you still experiencing this issue?” > > > - > > > > > > “What is the status of the implementation”? > > > - > > > > > >

Re: [DISCUSS] Feedback Collection Jira Bot

2021-06-29 Thread Stephan Ewen
Being a bit late to the party, and don't want to ask to change everything, just maybe some observation. My main observation and concern is still that this puts pressure and confusion on contributors, which are mostly blocked on committers for reviews, or are taking tickets as multi-week projects.

Re: [DISCUSS] FLIP-172: Support custom transactional.id prefix in FlinkKafkaProducer

2021-06-28 Thread Stephan Ewen
gt; > Thanks, > Wenhao > > On Wed, Jun 23, 2021 at 3:41 PM Piotr Nowojski > wrote: > > > > Hi, > > > > +1 from my side on this idea. I do not see any problems that could be > > caused by this change. > > > > Best, > > Piotrek > >

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Stephan Ewen
Thanks for writing this up, this also reflects my understanding. I think a blog post would be nice, ideally with an explicit call for feedback so we learn about user concerns. A blog post has a lot more reach than an ML thread. Best, Stephan On Wed, Jun 23, 2021 at 12:23 PM Timo Walther

Re: [DISCUSS] Drop Mesos in 1.14

2021-06-23 Thread Stephan Ewen
I would prefer to remove Mesos from the Flink core as well. I also had a similar thought as Seth: As far as I know, you can package applications to run on Mesos with "Marathon". That would be like deploying an opaque Flink standalone cluster on Mesos The implication is similar to going from an

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-23 Thread Stephan Ewen
+1 to Xintong's proposal On Wed, Jun 23, 2021 at 1:53 PM Till Rohrmann wrote: > I would first try to not introduce the exception for local builds. It makes > it quite hard for others to verify the build and to make sure that the > right things were executed. If we see that this becomes an issue

Re: [DISCUSS] FLIP-172: Support custom transactional.id prefix in FlinkKafkaProducer

2021-06-23 Thread Stephan Ewen
The motivation and the proposal sound good to me, +1 from my side. Would be good to have a quick opinion from someone who worked specifically with Kafka, maybe Becket or Piotr? Best, Stephan On Sat, Jun 12, 2021 at 9:50 AM Wenhao Ji wrote: > Hi everyone, > > I would like to open this

[jira] [Created] (FLINK-23093) Limit number of I/O pool and Future threads in Mini Cluster

2021-06-22 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23093: Summary: Limit number of I/O pool and Future threads in Mini Cluster Key: FLINK-23093 URL: https://issues.apache.org/jira/browse/FLINK-23093 Project: Flink

[jira] [Created] (FLINK-22753) Activate log-based Checkpoints for StateFun

2021-05-21 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22753: Summary: Activate log-based Checkpoints for StateFun Key: FLINK-22753 URL: https://issues.apache.org/jira/browse/FLINK-22753 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-22752) Add robust default state configuration to StateFun

2021-05-21 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22752: Summary: Add robust default state configuration to StateFun Key: FLINK-22752 URL: https://issues.apache.org/jira/browse/FLINK-22752 Project: Flink Issue

[jira] [Created] (FLINK-22751) Reduce JVM Metaspace memory for StateFun

2021-05-21 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22751: Summary: Reduce JVM Metaspace memory for StateFun Key: FLINK-22751 URL: https://issues.apache.org/jira/browse/FLINK-22751 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-22749) Apply a robust default State Backend Configuration

2021-05-21 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22749: Summary: Apply a robust default State Backend Configuration Key: FLINK-22749 URL: https://issues.apache.org/jira/browse/FLINK-22749 Project: Flink Issue

[jira] [Created] (FLINK-22741) Hide Flink complexity from Stateful Functions

2021-05-21 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22741: Summary: Hide Flink complexity from Stateful Functions Key: FLINK-22741 URL: https://issues.apache.org/jira/browse/FLINK-22741 Project: Flink Issue Type

Re: [Statefun] Truncated Messages in Python workers

2021-05-20 Thread Stephan Ewen
Thanks for reporting this, it looks indeed like a potential bug. I filed this Jira for it: https://issues.apache.org/jira/browse/FLINK-22729 Could you share (here ot in Jira) what the stack on the Python Worker side is (for example which HTTP server)? Do you know if the message truncation

[jira] [Created] (FLINK-22729) Truncated Messages in Python workers

2021-05-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22729: Summary: Truncated Messages in Python workers Key: FLINK-22729 URL: https://issues.apache.org/jira/browse/FLINK-22729 Project: Flink Issue Type: Bug

Re: [VOTE] Release 1.13.0, release candidate #2

2021-04-28 Thread Stephan Ewen
Flink framework. > > Sorry for the false alarm. > > Stephan Ewen 于2021年4月28日周三 下午3:23写道: > > > @Caizhi and @Becket - let me reach out to you to jointly debug this > issue. > > > > I am wondering if there is some incorrect reporting of failed events? > >

Re: [VOTE] Release 1.12.3, release candidate #1

2021-04-28 Thread Stephan Ewen
A quick heads-up: A fix from 1.13.0 that I backported to 1.12.3 is apparently causing some issues at larger batch scale. We are investigating this, but it would affect this release as well. Please see the mail thread "[VOTE] Release 1.13.0, release candidate #2" for details. If we want to make

Re: [VOTE] Release 1.13.0, release candidate #2

2021-04-28 Thread Stephan Ewen
@Caizhi and @Becket - let me reach out to you to jointly debug this issue. I am wondering if there is some incorrect reporting of failed events? On Wed, Apr 28, 2021 at 8:53 AM Caizhi Weng wrote: > -1 > > We're testing this version on batch jobs with large (600~1000) parallelisms > and the

[jira] [Created] (FLINK-22433) CoordinatorEventsExactlyOnceITCase stalls on Adaptive Scheduler

2021-04-23 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22433: Summary: CoordinatorEventsExactlyOnceITCase stalls on Adaptive Scheduler Key: FLINK-22433 URL: https://issues.apache.org/jira/browse/FLINK-22433 Project: Flink

[jira] [Created] (FLINK-22406) ReactiveModeITCase.testScaleDownOnTaskManagerLoss()

2021-04-22 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22406: Summary: ReactiveModeITCase.testScaleDownOnTaskManagerLoss() Key: FLINK-22406 URL: https://issues.apache.org/jira/browse/FLINK-22406 Project: Flink Issue

Re: Task Local Recovery with mountable disks in the cloud

2021-04-21 Thread Stephan Ewen
/cc dev@flink On Tue, Apr 20, 2021 at 1:29 AM Sonam Mandal wrote: > Hello, > > We've been experimenting with Task-local recovery using Kubernetes. We > have a way to specify mounting the same disk across Task Manager > restarts/deletions for when the pods get recreated. In this scenario, we >

[jira] [Created] (FLINK-22358) Add missing stability annotation to Split Reader API classes

2021-04-19 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22358: Summary: Add missing stability annotation to Split Reader API classes Key: FLINK-22358 URL: https://issues.apache.org/jira/browse/FLINK-22358 Project: Flink

[jira] [Created] (FLINK-22357) Mark FLIP-27 Source API as stable

2021-04-19 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22357: Summary: Mark FLIP-27 Source API as stable Key: FLINK-22357 URL: https://issues.apache.org/jira/browse/FLINK-22357 Project: Flink Issue Type: Task

Re: [DISCUSS]FLIP-150: Introduce Hybrid Source

2021-04-17 Thread Stephan Ewen
stood, significant portion of use cases. > > I'm planning to continue work on the prototype that I had shared. > There is production level usage waiting for it fairly soon. I expect > to open a PR in the coming weeks. > > Thomas > > > > > > On Tue, Apr 13, 20

[jira] [Created] (FLINK-22324) Backport FLINK-18071 for 1.12.x

2021-04-16 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22324: Summary: Backport FLINK-18071 for 1.12.x Key: FLINK-22324 URL: https://issues.apache.org/jira/browse/FLINK-22324 Project: Flink Issue Type: Bug

Re: [DISCUSS] Releasing Flink 1.12.3

2021-04-16 Thread Stephan Ewen
+1 Thanks for pushing this, Arvid, let's get this fix out asap. On Fri, Apr 16, 2021 at 9:46 AM Arvid Heise wrote: > Dear devs, > > Since we just fixed a severe bug that causes the dataflow to halt under > specific circumstances [1], we would like to release a bugfix asap. > > I would

Re: [DISCUSS]FLIP-150: Introduce Hybrid Source

2021-04-13 Thread Stephan Ewen
Thanks all for this discussion. Looks like there are lots of ideas and folks that are eager to do things, so let's see how we can get this moving. My take on this is the following: There will probably not be one Hybrid source, but possibly multiple ones, because of different

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-04-13 Thread Stephan Ewen
Hi all! Generally, avoiding API changes in Bug fix versions is the right thing, in my opinion. But this case is a bit special, because we are changing something that never worked properly in the first place. So we are not breaking a "running thing" here, but making it usable. So +1 from my side

[jira] [Created] (FLINK-22093) Unstable test "ThreadInfoSampleServiceTest.testShouldThrowExceptionIfTaskIsNotRunningBeforeSampling"

2021-04-01 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22093: Summary: Unstable test "ThreadInfoSampleServiceTest.testShouldThrowExceptionIfTaskIsNotRunningBeforeSampling" Key: FLINK-22093 URL: https://issues.apache.org/jira/br

[jira] [Created] (FLINK-22069) Check Log Pollution for 1.13 release

2021-03-31 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-22069: Summary: Check Log Pollution for 1.13 release Key: FLINK-22069 URL: https://issues.apache.org/jira/browse/FLINK-22069 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-21996) Transient RPC failure without TaskManager failure can lead to split assignment loss

2021-03-26 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-21996: Summary: Transient RPC failure without TaskManager failure can lead to split assignment loss Key: FLINK-21996 URL: https://issues.apache.org/jira/browse/FLINK-21996

[jira] [Created] (FLINK-21935) Remove "state.backend.async" option.

2021-03-23 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-21935: Summary: Remove "state.backend.async" option. Key: FLINK-21935 URL: https://issues.apache.org/jira/browse/FLINK-21935 Project: Flink Issue Type: I

Re: Re: Re: [DISCUSSION] Introduce a separated memory pool for the TM merge shuffle

2021-03-22 Thread Stephan Ewen
> 发件人:Guowei Ma > 日 期:2021年03月09日 17:28:35 > 收件人:曹英杰(北牧) > 抄 送:Till Rohrmann; Stephan Ewen; > dev; user; Xintong Song< > tonysong...@gmail.com> > 主 题:Re: Re: [DISCUSSION] Introduce a separated memory pool for the TM > merge shuffle > > Hi, all &

Re: [ANNOUCE][DISCUSS] Roadmap Update for Website

2021-03-15 Thread Stephan Ewen
I created a Pull Request for the roadmap update: https://github.com/apache/flink-web/pull/426 Happy to take reviews! Best, Stephan On Wed, Mar 10, 2021 at 1:55 PM Stephan Ewen wrote: > Thanks for the positive feedback. Will convert this to a pull request then > in the next days. >

Re: [ANNOUCE][DISCUSS] Roadmap Update for Website

2021-03-10 Thread Stephan Ewen
see the > roadmap updated. I also find the "Feature Radar" extremely helpful. It > looks good to be published from my side. > > Best, > > Dawid > > On 02/03/2021 15:38, Stephan Ewen wrote: > > Hi all! > > > > The roadmap on the Flink website is qu

[jira] [Created] (FLINK-21695) Increase default value for number of KeyGroups

2021-03-09 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-21695: Summary: Increase default value for number of KeyGroups Key: FLINK-21695 URL: https://issues.apache.org/jira/browse/FLINK-21695 Project: Flink Issue Type

[jira] [Created] (FLINK-21694) Increase default value of "state.backend.rocksdb.checkpoint.transfer.thread.num"

2021-03-09 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-21694: Summary: Increase default value of "state.backend.rocksdb.checkpoint.transfer.thread.num" Key: FLINK-21694 URL: https://issues.apache.org/jira/browse/F

Re: [ANNOUCE][DISCUSS] Roadmap Update for Website

2021-03-08 Thread Stephan Ewen
ally like the new Feature Radar, because it makes some assumptions > floating around on JIRA and the lists properly documented. > > I guess the radar is missing the Reactive Mode as an MVP feature. > > > Otherwise, I'm +1 on publishing this soon! > > On Tue, Mar 2, 2021 at 3:38

Re: [DISCUSSION] Introduce a separated memory pool for the TM merge shuffle

2021-03-06 Thread Stephan Ewen
Thanks Guowei, for the proposal. As discussed offline already, I think this sounds good. One thought is that 16m sounds very small for a default read buffer pool. How risky do you think it is to increase this to 32m or 64m? Best, Stephan On Fri, Mar 5, 2021 at 4:32 AM Guowei Ma wrote: > Hi,

Re: [DISCUSSION] Introduce a separated memory pool for the TM merge shuffle

2021-03-05 Thread Stephan Ewen
Thanks Guowei, for the proposal. As discussed offline already, I think this sounds good. One thought is that 16m sounds very small for a default read buffer pool. How risky do you think it is to increase this to 32m or 64m? Best, Stephan On Fri, Mar 5, 2021 at 4:33 AM Guowei Ma wrote: > Hi,

[ANNOUCE][DISCUSS] Roadmap Update for Website

2021-03-02 Thread Stephan Ewen
Hi all! The roadmap on the Flink website is quite outdated: https://flink.apache.org/roadmap.html I drafted an update to the roadmap that reflects the currently ongoing bigger threads. Not every detail is mentioned there, because this roadmap should give users a high-level view where the project

Re: [DISCUSS] FLIP-151: Incremental snapshots for heap-based state backend

2021-02-16 Thread Stephan Ewen
ar: ChangelogStateBackend has to encode > removal operations and send them to StateChangelog (though no additional > data structure is required). > > Regards, > Roman > > > On Thu, Feb 11, 2021 at 4:28 PM Stephan Ewen wrote: > > > Thanks, Roman for publishing this design.

Re: [DISCUSS] FLIP-151: Incremental snapshots for heap-based state backend

2021-02-11 Thread Stephan Ewen
Thanks, Roman for publishing this design. There seems to be quite a bit of overlap with FLIP-158 (generalized incremental checkpoints). I would go with +1 to the effort if it is a pretty self-contained and closed effort. Meaning we don't expect that this needs a ton of follow-ups, other than

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-02-03 Thread Stephan Ewen
Hi all! A quick thought on this thread: We see a typical stalemate here, as in so many discussions recently. One developer prefers it this way, another one another way. Both have pro/con arguments, it takes a lot of time from everyone, still there is little progress in the discussion.

Re: [DISCUSS] FLIP-158: Generalized incremental checkpoints

2021-01-28 Thread Stephan Ewen
+1 to this FLIP in general. I like the general idea very much (full disclosure, have been involved in the discussions and drafting of the design for a while, so I am not a new/neutral reviewer here). One thing I would like to see us do here, is reaching out to users early with this, and

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

2021-01-18 Thread Stephan Ewen
Thanks a lot, Yangze and Xintong for this FLIP. I want to say, first of all, that this is super well written. And the points that the FLIP makes about how to expose the configuration to users is exactly the right thing to figure out first. So good job here! About how to let users specify the

Re: [DISCUSS] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

2021-01-17 Thread Stephan Ewen
+1 for this FLIP. I tried it out locally, and it works well. Aside from the fact build times, I really like the search feature, which works pretty well. Thanks a lot for the initiative, Seth! On Sun, Jan 17, 2021 at 4:43 PM Jark Wu wrote: > I have tried in my local env, and the build time is

[jira] [Created] (FLINK-20472) Change quickstarts to have a dependency on "flink-dist"

2020-12-03 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20472: Summary: Change quickstarts to have a dependency on "flink-dist" Key: FLINK-20472 URL: https://issues.apache.org/jira/browse/FLINK-20472 Proj

[jira] [Created] (FLINK-20413) Sources should add splits back in "resetSubtask()", rather than in "subtaskFailed()".

2020-11-29 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20413: Summary: Sources should add splits back in "resetSubtask()", rather than in "subtaskFailed()". Key: FLINK-20413 URL: https://issues.apache.org/jir

[jira] [Created] (FLINK-20412) Collect Result Fetching occasionally fails after a JobManager Failover

2020-11-29 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20412: Summary: Collect Result Fetching occasionally fails after a JobManager Failover Key: FLINK-20412 URL: https://issues.apache.org/jira/browse/FLINK-20412 Project

[jira] [Created] (FLINK-20406) Return the Checkpoint ID of the restored Checkpoint in CheckpointCoordinator.restoreLatestCheckpointedStateToSubtasks()

2020-11-27 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20406: Summary: Return the Checkpoint ID of the restored Checkpoint in CheckpointCoordinator.restoreLatestCheckpointedStateToSubtasks() Key: FLINK-20406 URL: https://issues.apache.org

[jira] [Created] (FLINK-20397) Pass checkpointId to OperatorCoordinator.resetToCheckpoint().

2020-11-27 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20397: Summary: Pass checkpointId to OperatorCoordinator.resetToCheckpoint(). Key: FLINK-20397 URL: https://issues.apache.org/jira/browse/FLINK-20397 Project: Flink

[jira] [Created] (FLINK-20396) Replace "OperatorCoordinator.subtaskFailed()" with "subtaskRestored()"

2020-11-27 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20396: Summary: Replace "OperatorCoordinator.subtaskFailed()" with "subtaskRestored()" Key: FLINK-20396 URL: https://issues.apache.org/jira/browse/FLINK-20396

[jira] [Created] (FLINK-20379) New Kafka Connector does not support DeserializationSchema

2020-11-26 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20379: Summary: New Kafka Connector does not support DeserializationSchema Key: FLINK-20379 URL: https://issues.apache.org/jira/browse/FLINK-20379 Project: Flink

[jira] [Created] (FLINK-20276) Transparent DeCompression of streams missing on new File Source

2020-11-22 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20276: Summary: Transparent DeCompression of streams missing on new File Source Key: FLINK-20276 URL: https://issues.apache.org/jira/browse/FLINK-20276 Project: Flink

[jira] [Created] (FLINK-20188) Add Documentation for new File Source

2020-11-17 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20188: Summary: Add Documentation for new File Source Key: FLINK-20188 URL: https://issues.apache.org/jira/browse/FLINK-20188 Project: Flink Issue Type: Task

Re: [VOTE] Apache Flink Stateful Functions 2.2.1, release candidate #1

2020-11-10 Thread Stephan Ewen
+1 (binding) Compiled source and ran all tests (JDK 8) Ran example (greeter) with a docker compose setup Release contains no binaries. Checked LICENSE and NOTICE for source release Checked LICENSE and NOTICE for flink-statefun-distribution binary release On Fri, Nov 6, 2020 at 11:46 PM Igal

[jira] [Created] (FLINK-20063) File Source requests an additional split on every restore.

2020-11-09 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20063: Summary: File Source requests an additional split on every restore. Key: FLINK-20063 URL: https://issues.apache.org/jira/browse/FLINK-20063 Project: Flink

[jira] [Created] (FLINK-20049) Simplify handling of "request split".

2020-11-08 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-20049: Summary: Simplify handling of "request split". Key: FLINK-20049 URL: https://issues.apache.org/jira/browse/FLINK-20049 Project: Flink

  1   2   3   4   5   6   7   8   9   10   >