[jira] [Created] (FLINK-8601) Introduce LinkedBloomFilterState for Approximate calculation and other situations of performance optimization

Sihua Zhou created FLINK-8601: - Summary: Introduce LinkedBloomFilterState for Approximate calculation and other situations of performance optimization Key: FLINK-8601 URL:

Re: build/devops

Generally we run the full test suite on travis for every pull request. I would guess that dev working on some part only run certain tests locally though. On 08.02.2018 01:06, cw7k wrote: Hi, a full mvn install takes about 2-2.5 hours. Does the Flink team run the full testsuite on every dev

Re: Timestamp/watermark support in Kinesis consumer

Generalizing the pattern would be great. I was also wondering if there aren't other commonalities between sources that would benefit from a shared framework. Kafka and Kinesis don't look all that different from a consumer perspective: replayable source, topic -> stream, partition -> shard, offset

build/devops

Hi, a full mvn install takes about 2-2.5 hours.  Does the Flink team run the full testsuite on every dev pull request, or use a main dev branch that gets the full test periodically?  Or a different method?  The 2 hours can be a drag when one team needs another's changes and each attempt at

[jira] [Created] (FLINK-8600) BucketingSink errors out when used with Presto filesystem

Narayanan Arunachalam created FLINK-8600: Summary: BucketingSink errors out when used with Presto filesystem Key: FLINK-8600 URL: https://issues.apache.org/jira/browse/FLINK-8600 Project:

[jira] [Created] (FLINK-8599) Improve the failure behavior of the ContinuousFileReaderOperator for bad files

Chengzhi Zhao created FLINK-8599: Summary: Improve the failure behavior of the ContinuousFileReaderOperator for bad files Key: FLINK-8599 URL: https://issues.apache.org/jira/browse/FLINK-8599

Re: [Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

Hi Bowen, Thanks for the proposal! I think state TTL would be a great feature! Actually, we have implemented this for SQL / Table API [1]. I've added a couple of comments to the design doc. In principle, I'm not sure if this functionality should be added to the state backends. We could also use

Re: Terminating streaming test

Thanks! It would indeed be nice to have this as framework that makes test fun and easy to write ;-) Looking at SavepointMigrationTestBase, I see that the cluster is eventually stopped in teardown, but I don't find where the individual job is terminated after the expected results are in? Also,

Re: [DISCUSS] Releasing Flink 1.4.1

Status update: We're waiting for the PR for [FLINK-8318] Shade all Elasticsearch connector dependencies to be reviewed and merged [1]. Once that happens, I’ll kick off RC1 for Flink 1.4.1. Cheers, Gordon [1] https://github.com/apache/flink/pull/5426 On 31 January 2018 at 12:15:09 PM, Tzu-Li

[jira] [Created] (FLINK-8598) Intermittent test failure of BackPressureStatsTrackerITCase.testBackPressuredProducer

Bowen Li created FLINK-8598: --- Summary: Intermittent test failure of BackPressureStatsTrackerITCase.testBackPressuredProducer Key: FLINK-8598 URL: https://issues.apache.org/jira/browse/FLINK-8598 Project:

[jira] [Created] (FLINK-8597) Add examples for Connected Streams with Broadcast State.

Kostas Kloudas created FLINK-8597: - Summary: Add examples for Connected Streams with Broadcast State. Key: FLINK-8597 URL: https://issues.apache.org/jira/browse/FLINK-8597 Project: Flink

Re: Timestamp/watermark support in Kinesis consumer

Hi, That last point is very valid. For a while now I've wanted to generalise the pattern of our file source to other sources. (This is related to how Beam sources are being refactored to use Splittable DoFn.) I'm very eager for design work to start on this once 1.5 is out the door. There are

Re: Terminating streaming test

There is StatefulJobSavepointMigrationITCase, which executes a proper unbounded pipeline on a locally started cluster and "listens" for some criteria via accumulators before cancelling the job and shutting down the cluster. The communication with the cluster is quite custom here, but I would

[jira] [Created] (FLINK-8596) Custom command line code does not correctly catch errors

Aljoscha Krettek created FLINK-8596: --- Summary: Custom command line code does not correctly catch errors Key: FLINK-8596 URL: https://issues.apache.org/jira/browse/FLINK-8596 Project: Flink

[jira] [Created] (FLINK-8595) Include table source factory services in flink-table jar

Timo Walther created FLINK-8595: --- Summary: Include table source factory services in flink-table jar Key: FLINK-8595 URL: https://issues.apache.org/jira/browse/FLINK-8595 Project: Flink Issue

[jira] [Created] (FLINK-8594) Make producesUpdates in DataStreamGroupAggregate return false if it is a distinct group by without state retention configuration

Hequn Cheng created FLINK-8594: -- Summary: Make producesUpdates in DataStreamGroupAggregate return false if it is a distinct group by without state retention configuration Key: FLINK-8594 URL:

[jira] [Created] (FLINK-8593) Latency metric docs are outdated

Chesnay Schepler created FLINK-8593: --- Summary: Latency metric docs are outdated Key: FLINK-8593 URL: https://issues.apache.org/jira/browse/FLINK-8593 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-8592) LatencyMetric scope should include operator names

Chesnay Schepler created FLINK-8592: --- Summary: LatencyMetric scope should include operator names Key: FLINK-8592 URL: https://issues.apache.org/jira/browse/FLINK-8592 Project: Flink Issue

[jira] [Created] (FLINK-8591) Handle unfinished BufferConsumers in subpartitions

Piotr Nowojski created FLINK-8591: - Summary: Handle unfinished BufferConsumers in subpartitions Key: FLINK-8591 URL: https://issues.apache.org/jira/browse/FLINK-8591 Project: Flink Issue

[jira] [Created] (FLINK-8590) Drop ResultPartitionWriter#writeBufferToAllSubpartitions method

Piotr Nowojski created FLINK-8590: - Summary: Drop ResultPartitionWriter#writeBufferToAllSubpartitions method Key: FLINK-8590 URL: https://issues.apache.org/jira/browse/FLINK-8590 Project: Flink

[jira] [Created] (FLINK-8589) Add data polling method to InputGate

Piotr Nowojski created FLINK-8589: - Summary: Add data polling method to InputGate Key: FLINK-8589 URL: https://issues.apache.org/jira/browse/FLINK-8589 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-8588) Handle sliced buffers in RecordDeserializer

Piotr Nowojski created FLINK-8588: - Summary: Handle sliced buffers in RecordDeserializer Key: FLINK-8588 URL: https://issues.apache.org/jira/browse/FLINK-8588 Project: Flink Issue Type:

[jira] [Created] (FLINK-8587) Drop unused AdaptiveSpanningRecordDeserializer

Piotr Nowojski created FLINK-8587: - Summary: Drop unused AdaptiveSpanningRecordDeserializer Key: FLINK-8587 URL: https://issues.apache.org/jira/browse/FLINK-8587 Project: Flink Issue Type:

[jira] [Created] (FLINK-8586) Drop SpilledSubpartitionViewTest

Piotr Nowojski created FLINK-8586: - Summary: Drop SpilledSubpartitionViewTest Key: FLINK-8586 URL: https://issues.apache.org/jira/browse/FLINK-8586 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-8585) Efficiently handle read-only buffers in DataInputDeserializer

Piotr Nowojski created FLINK-8585: - Summary: Efficiently handle read-only buffers in DataInputDeserializer Key: FLINK-8585 URL: https://issues.apache.org/jira/browse/FLINK-8585 Project: Flink

[jira] [Created] (FLINK-8584) Handle read-only buffers in DataInputDeserializer

Piotr Nowojski created FLINK-8584: - Summary: Handle read-only buffers in DataInputDeserializer Key: FLINK-8584 URL: https://issues.apache.org/jira/browse/FLINK-8584 Project: Flink Issue

[jira] [Created] (FLINK-8583) Pass BufferConsumer to subpartitions

Piotr Nowojski created FLINK-8583: - Summary: Pass BufferConsumer to subpartitions Key: FLINK-8583 URL: https://issues.apache.org/jira/browse/FLINK-8583 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-8582) Introduce BufferConsumer concept

Piotr Nowojski created FLINK-8582: - Summary: Introduce BufferConsumer concept Key: FLINK-8582 URL: https://issues.apache.org/jira/browse/FLINK-8582 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-8581) Improve performance for low latency network

Piotr Nowojski created FLINK-8581: - Summary: Improve performance for low latency network Key: FLINK-8581 URL: https://issues.apache.org/jira/browse/FLINK-8581 Project: Flink Issue Type:

[jira] [Created] (FLINK-8580) No easy way (or issues when trying?) to handle multiple yarn sessions and choose at runtime the one to submit a ha streaming job

Arnaud Linz created FLINK-8580: -- Summary: No easy way (or issues when trying?) to handle multiple yarn sessions and choose at runtime the one to submit a ha streaming job Key: FLINK-8580 URL:

[jira] [Created] (FLINK-8579) Support deletion flag for DataStream to Table upsert conversion.

Hequn Cheng created FLINK-8579: -- Summary: Support deletion flag for DataStream to Table upsert conversion. Key: FLINK-8579 URL: https://issues.apache.org/jira/browse/FLINK-8579 Project: Flink

[jira] [Created] (FLINK-8578) Implement rowtime DataStream to Table upsert conversion.

Hequn Cheng created FLINK-8578: -- Summary: Implement rowtime DataStream to Table upsert conversion. Key: FLINK-8578 URL: https://issues.apache.org/jira/browse/FLINK-8578 Project: Flink Issue

[jira] [Created] (FLINK-8577) Implement proctime DataStream to Table upsert conversion.

Hequn Cheng created FLINK-8577: -- Summary: Implement proctime DataStream to Table upsert conversion. Key: FLINK-8577 URL: https://issues.apache.org/jira/browse/FLINK-8577 Project: Flink Issue

Re: [DISCUSS] Releasing Flink 1.5.0

As Aljoscha said we wanted to do 1.5 soon after 1.4 based on the assumption that the 3 big features (FLIP-6, network stack changes, local state recovery) are nearly done. I'm unsure about local state recovery, but I still see open issues for FLIP-6 and the network stack rework. As such it

[jira] [Created] (FLINK-8576) Log message for QueryableState loading failure too verbose

Chesnay Schepler created FLINK-8576: --- Summary: Log message for QueryableState loading failure too verbose Key: FLINK-8576 URL: https://issues.apache.org/jira/browse/FLINK-8576 Project: Flink

[jira] [Created] (FLINK-8575) BackPressureStatsTrackerITCase unstable

Chesnay Schepler created FLINK-8575: --- Summary: BackPressureStatsTrackerITCase unstable Key: FLINK-8575 URL: https://issues.apache.org/jira/browse/FLINK-8575 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-8574) Add timestamps to travis logging messages

Chesnay Schepler created FLINK-8574: --- Summary: Add timestamps to travis logging messages Key: FLINK-8574 URL: https://issues.apache.org/jira/browse/FLINK-8574 Project: Flink Issue Type: