Re: FLINK-9752 (s3 recoverable writer) not actually fixed in 1.6.2?

I have backported this at https://github.com/instructure/flink/tree/s3_recover_backport by cherry-picking all the relevant code, I am not sure how backports are usually done with Flink (if you squash and merge) but there were a few minor conflicts and involved quite a few changes from master.

Re: FLINK-9752 (s3 recoverable writer) not actually fixed in 1.6.2?

FWIW I also tried this on Flink 1.6.2 today and got the same error. This is my full stack trace: java.lang.UnsupportedOperationException: Recoverable writers on Hadoop are only supported for HDFS and for Hadoop version 2.7 or newer at

Run Flink Natively On Kubernetes

Hi all, FLIP-6 outlines improvements in how Flink is used on top of various cluster managers. we already have implementation on Yarn and Mesos, however, no native implementation on K8s. As a follow up, FLINK-9953 was opened by Till to track

[jira] [Created] (FLINK-10729) Create a Hive a connector to access Hive data

Xuefu Zhang created FLINK-10729: --- Summary: Create a Hive a connector to access Hive data Key: FLINK-10729 URL: https://issues.apache.org/jira/browse/FLINK-10729 Project: Flink Issue Type:

Re: FLINK-9752 (s3 recoverable writer) not actually fixed in 1.6.2?

Hi Addison, I think the idea was to also backport this feature to 1.6 since we considered it a bug that S3 was not supported in 1.6. I've pulled in Kostas who worked on the S3 writer. @Klou did we intentionally not backport this feature? I think there should be nothing special about backporting

FLINK-9752 (s3 recoverable writer) not actually fixed in 1.6.2?

Hi all, Been hitting my head against a wall for the last few hours. The release notes for 1.6.2 show https://issues.apache.org/jira/browse/FLINK-9752 as resolved in 1.6.2. I am trying to upgrade and switch some things to use the StreamingFileSink against s3. However, when doing so, I get the

[jira] [Created] (FLINK-10728) Unexpected items on ASF mirrors

Sebb created FLINK-10728: Summary: Unexpected items on ASF mirrors Key: FLINK-10728 URL: https://issues.apache.org/jira/browse/FLINK-10728 Project: Flink Issue Type: Bug Environment:

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

Hi all, I have also shared a design doc on Hive metastore integration that is attached here and also to FLINK-10556[1]. Please kindly review and share your feedback. Thanks, Xuefu [1] https://issues.apache.org/jira/browse/FLINK-10556

Re: [ANNOUNCE] Apache Flink 1.5.5 released

No this wasn't intentional. This should actually be quite a big problem since force-shading is a dependency of all modules; if it is missing users cannot compile against 1.5.5 dependencies. I don't think this has happened before and am rather puzzled how it can, will investigate more

[jira] [Created] (FLINK-10727) Remove unnecessary synchronization in SingleInputGate#requestPartitions()

Nico Kruber created FLINK-10727: --- Summary: Remove unnecessary synchronization in SingleInputGate#requestPartitions() Key: FLINK-10727 URL: https://issues.apache.org/jira/browse/FLINK-10727 Project:

[ANNOUNCE] Weekly community update #44

Dear community, this is the weekly community update thread #44. Please post any news and updates you want to share with the community to this thread. # Flink 1.5.5 and 1.6.2 have been released The Flink community is proud to announce the two bugfix release Flink 1.5.5 [1] and Flink 1.6.2 [2]

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

Thanks for starting this discussion Fabian! I think our web UI technology stack is quite dusty by now and it would be beneficial to think about its technological future. On the one hand, our current web UI works more or less reliable and changing the underlying technology has the risk of breaking

Re: [DISCUSS] Breaking the Scala API for Scala 2.12 Support

Just to clarify, this is an example of a change I had to do to get the code working with Scala 2.12. With Scala 2.11 this works: val terminate = prevPaths .coGroup(nextPaths) .where(0).equalTo(0) { (prev, next, out: Collector[(Long, Long)]) => { val prevPaths = prev.toSet for

[jira] [Created] (FLINK-10726) Streaming SQL end-to-end test fails due to recent flink-table-common changes

Timo Walther created FLINK-10726: Summary: Streaming SQL end-to-end test fails due to recent flink-table-common changes Key: FLINK-10726 URL: https://issues.apache.org/jira/browse/FLINK-10726

[jira] [Created] (FLINK-10725) Support for Java 11 (LTS)

Sina Madani created FLINK-10725: --- Summary: Support for Java 11 (LTS) Key: FLINK-10725 URL: https://issues.apache.org/jira/browse/FLINK-10725 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-10724) Refactor failure handling in check point coordinator

Andrey Zagrebin created FLINK-10724: --- Summary: Refactor failure handling in check point coordinator Key: FLINK-10724 URL: https://issues.apache.org/jira/browse/FLINK-10724 Project: Flink

Re: [DISCUSS] Breaking the Scala API for Scala 2.12 Support

I think there was no conclusion so I didn't go for breaking the API in the end. What this means for users is: - when using Scala 2.11 nothing changes - you can now use Scala 2.12, but you might have to add explicit types to disambiguate calls. This disambiguation is not needed when using

Re: [ANNOUNCE] Apache Flink 1.5.5 released

Great work! :) Was just trying the new release out with Beam and found that "force-shading" is not published for 1.5.5. Also visible here: https://repo.maven.apache.org/maven2/org/apache/flink/force-shading/ Was this intentional? It is not a big problem because it is a dependency which is

[jira] [Created] (FLINK-10723) Upgrade MapSerializer for state evolution

Tzu-Li (Gordon) Tai created FLINK-10723: --- Summary: Upgrade MapSerializer for state evolution Key: FLINK-10723 URL: https://issues.apache.org/jira/browse/FLINK-10723 Project: Flink

[jira] [Created] (FLINK-10722) Document MATCH_RECOGNIZE

Dawid Wysakowicz created FLINK-10722: Summary: Document MATCH_RECOGNIZE Key: FLINK-10722 URL: https://issues.apache.org/jira/browse/FLINK-10722 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10721) kafkaFetcher runFetchLoop throw exception will cause follow-up code not execute in FlinkKafkaConsumerBase run method

zhaoshijie created FLINK-10721: -- Summary: kafkaFetcher runFetchLoop throw exception will cause follow-up code not execute in FlinkKafkaConsumerBase run method Key: FLINK-10721 URL: