[jira] [Created] (FLINK-10559) Remove LegacyLocalStreamEnvironment

2018-10-15 Thread TisonKun (JIRA)
TisonKun created FLINK-10559: Summary: Remove LegacyLocalStreamEnvironment Key: FLINK-10559 URL: https://issues.apache.org/jira/browse/FLINK-10559 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10558) Port YARNHighAvailabilityITCase and YARNSessionCapacitySchedulerITCase to new code base

2018-10-15 Thread vinoyang (JIRA)
vinoyang created FLINK-10558: Summary: Port YARNHighAvailabilityITCase and YARNSessionCapacitySchedulerITCase to new code base Key: FLINK-10558 URL: https://issues.apache.org/jira/browse/FLINK-10558

Re: [DISCUSS] Release 1.5.5 and 1.6.2

2018-10-15 Thread Till Rohrmann
FLINK-9932 has been merged. Cheers, Till On Mon, Oct 15, 2018 at 1:19 PM Till Rohrmann wrote: > Thanks a lot for starting this discussion Chesnay. I fully agree that a > new bug fix release would be justified. > > I'm currently working on FLINK-9932 which I would like to include in the > next

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-10-15 Thread Zhang, Xuefu
Hi Bowen, Thank you for your feedback and interest in the project. Your contribution is certainly welcome. Per your suggestion, I have created an Uber JIRA (https://issues.apache.org/jira/browse/FLINK-10556) to track our overall effort on this. For each subtask, we'd like to see a short

[jira] [Created] (FLINK-10556) Integration with Apache Hive

2018-10-15 Thread Xuefu Zhang (JIRA)
Xuefu Zhang created FLINK-10556: --- Summary: Integration with Apache Hive Key: FLINK-10556 URL: https://issues.apache.org/jira/browse/FLINK-10556 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-10555) Port AkkaSslITCase to new code base

2018-10-15 Thread TisonKun (JIRA)
TisonKun created FLINK-10555: Summary: Port AkkaSslITCase to new code base Key: FLINK-10555 URL: https://issues.apache.org/jira/browse/FLINK-10555 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10554) Bump flink-shaded dependency version

2018-10-15 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10554: Summary: Bump flink-shaded dependency version Key: FLINK-10554 URL: https://issues.apache.org/jira/browse/FLINK-10554 Project: Flink Issue Type:

[jira] [Created] (FLINK-10553) Unified table sink and source name in SQL statement

2018-10-15 Thread wangsan (JIRA)
wangsan created FLINK-10553: --- Summary: Unified table sink and source name in SQL statement Key: FLINK-10553 URL: https://issues.apache.org/jira/browse/FLINK-10553 Project: Flink Issue Type: Bug

[RESULT][VOTE] Release flink-shaded 5.0, release candidate #1

2018-10-15 Thread Chesnay Schepler
|I'm happy to announce that we have unanimously approved this release.| |There are 4 approving votes, 3 of which are binding:| |* Aljoscha (binding) | |* Timo (binding) | |* Gordin (binding) | |* vino (non-binding) | |There are no disapproving votes.| |Thanks everyone!| On 10.10.2018 15:11,

Re: Add checkpoint hooks to window triggers

2018-10-15 Thread Fabian Hueske
Hi Paul, I think this would be very tricky to implement and interfere with many parts of the system like state backends, checkpointing logic, etc. We would need to maintain a copy (or version) of the state at the time of a checkpoint. There might be multiple checkpoints in flight. Checkpoints

Re: Add checkpoint hooks to window triggers

2018-10-15 Thread Paul Lam
Hi Fabian, Perhaps I didn’t explain that clearly. Actually I want a trigger to fire when a checkpoint is completed, and emit the intermediate results in consistency with the completed checkpoint. It works like this: 1) Once the window operator receives a barrier, it performs the snapshot as

Re: [DISCUSS] Release 1.5.5 and 1.6.2

2018-10-15 Thread Till Rohrmann
Thanks a lot for starting this discussion Chesnay. I fully agree that a new bug fix release would be justified. I'm currently working on FLINK-9932 which I would like to include in the next bug fix release. It should be done by the end of today. Cheers, Till On Mon, Oct 15, 2018 at 11:40 AM

[DISCUSS] Release 1.5.5 and 1.6.2

2018-10-15 Thread Chesnay Schepler
Hello, we've accumulated various fixes for 1.5.5 (24) and 1.6.2 (37) that improve stability by quite a bit along with some neat usability fixes. I'm proposing to do the next bugfix releases soon (I suggest tomorrow as a tentative vote date), and volunteer to handle the release process for

Re: Please review : Re: PubSub connector (FLINK-9311)

2018-10-15 Thread Richard Deurwaarder
Hello everyone, To improve performance we have changed some parts of the code: * Previously the same threads used to poll PubSub we're used to actually run the flink pipeline, this caused some issues in the PubSub SDK. We've changed this so that the PubSub threads are only used to poll PubSub and

Re: Handling burst I/O when using tumbling/sliding windows

2018-10-15 Thread Piotr Nowojski
Hi, Just to sum up this thread. Discussion in the before mentioned design doc concluded that there is no need for API changes. The semantic that Rong was asking for can be achieved by implementing a custom WindowAssigner, that mimics one of the existing ones like TumblingEventTimeWindows, and

[jira] [Created] (FLINK-10552) Provide RichAsyncFunction for scala API

2018-10-15 Thread Shimin Yang (JIRA)
Shimin Yang created FLINK-10552: --- Summary: Provide RichAsyncFunction for scala API Key: FLINK-10552 URL: https://issues.apache.org/jira/browse/FLINK-10552 Project: Flink Issue Type: Bug

Re: Add checkpoint hooks to window triggers

2018-10-15 Thread Fabian Hueske
Hi Paul, If I got your proposal right, you'd like to fire a Trigger right before a checkpoint is taken, correct? So, before taking a checkpoint, a Trigger would fire and the operator would process and emit some intermediate results. This approach would not completely solve the consistency issue

[jira] [Created] (FLINK-10551) Remove legacy REST handlers

2018-10-15 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10551: Summary: Remove legacy REST handlers Key: FLINK-10551 URL: https://issues.apache.org/jira/browse/FLINK-10551 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10550) Remove LegacyYarnClusterDescriptor

2018-10-15 Thread vinoyang (JIRA)
vinoyang created FLINK-10550: Summary: Remove LegacyYarnClusterDescriptor Key: FLINK-10550 URL: https://issues.apache.org/jira/browse/FLINK-10550 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10549) Remove LegacyJobRetrievalITCase

2018-10-15 Thread TisonKun (JIRA)
TisonKun created FLINK-10549: Summary: Remove LegacyJobRetrievalITCase Key: FLINK-10549 URL: https://issues.apache.org/jira/browse/FLINK-10549 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10548) Remove LegacyStandaloneClusterDescriptor

2018-10-15 Thread vinoyang (JIRA)
vinoyang created FLINK-10548: Summary: Remove LegacyStandaloneClusterDescriptor Key: FLINK-10548 URL: https://issues.apache.org/jira/browse/FLINK-10548 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10547) Remove LegacyCLI

2018-10-15 Thread vinoyang (JIRA)
vinoyang created FLINK-10547: Summary: Remove LegacyCLI Key: FLINK-10547 URL: https://issues.apache.org/jira/browse/FLINK-10547 Project: Flink Issue Type: Sub-task Reporter: vinoyang