[jira] [Reopened] (FLINK-8978) End-to-end test: Job upgrade

[ https://issues.apache.org/jira/browse/FLINK-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter reopened FLINK-8978: --- Change fix version > End-to-end test: Job upgr

[jira] [Closed] (FLINK-8978) End-to-end test: Job upgrade

[ https://issues.apache.org/jira/browse/FLINK-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8978. - Resolution: Fixed Merged in: master: 5ac4d29609 release-1.5: 54befe5a31 > End-to-end test:

Re: This server is not the leader for that topic-partition

Hi, I think in general this means that your producer client does not connect to the correct Broker (the leader) but to a broker that is just a follower and the follower can not execute that request. However, I am not sure what causes this in the context of the FlinkKafkaProducer. Maybe Piotr

[jira] [Closed] (FLINK-9254) Move NotSoMiniClusterIterations to be an end-to-end test

[ https://issues.apache.org/jira/browse/FLINK-9254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-9254. - Resolution: Fixed Merged in: master: b50cebb656 release-1.5: ed3447e343 > M

Re: Flink + Marathon (Mesos) Memory Issues

Hi, besides your configured heap size, there is also some off-heap memory used in the JVM process, in particular by RocksDB. Each keyed operator instance on a TM has its own RocksDB instance, so the question is how many are running in one container and what is their configuration? For RocksDB

[jira] [Closed] (FLINK-9269) Concurrency problem in HeapKeyedStateBackend when performing checkpoint async

[ https://issues.apache.org/jira/browse/FLINK-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-9269. - Resolution: Fixed Merged in: master: 14e7d35f26 release-1.5: 3ba21adc0e > Concurrency prob

[jira] [Commented] (FLINK-9268) RockDB errors from WindowOperator

[ https://issues.apache.org/jira/browse/FLINK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462164#comment-16462164 ] Stefan Richter commented on FLINK-9268: --- It is hard to make any assumptions about your job without

[jira] [Commented] (FLINK-9268) RockDB errors from WindowOperator

[ https://issues.apache.org/jira/browse/FLINK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462086#comment-16462086 ] Stefan Richter commented on FLINK-9268: --- Yes, you could have duplicates in AT_LEAST_ONCE

[jira] [Issue Comment Deleted] (FLINK-9268) RockDB errors from WindowOperator

[ https://issues.apache.org/jira/browse/FLINK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter updated FLINK-9268: -- Comment: was deleted (was: Maybe what your are looking for is using a window with aggregate

[jira] [Commented] (FLINK-9268) RockDB errors from WindowOperator

[ https://issues.apache.org/jira/browse/FLINK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462072#comment-16462072 ] Stefan Richter commented on FLINK-9268: --- Maybe what your are looking for is using a window

[jira] [Commented] (FLINK-9268) RockDB errors from WindowOperator

[ https://issues.apache.org/jira/browse/FLINK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462068#comment-16462068 ] Stefan Richter commented on FLINK-9268: --- The 2GB limit actually applies on a per-key-per-window

[jira] [Commented] (FLINK-9291) Checkpoint failure (CIRCULAR REFERENCE:java.lang.NegativeArraySizeException)

[ https://issues.apache.org/jira/browse/FLINK-9291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462025#comment-16462025 ] Stefan Richter commented on FLINK-9291: --- Yes, this is basically a duplicate, just the access

[jira] [Commented] (FLINK-9268) RockDB errors from WindowOperator

[ https://issues.apache.org/jira/browse/FLINK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461231#comment-16461231 ] Stefan Richter commented on FLINK-9268: --- I don't think that is the cause. The exception comes right

[jira] [Commented] (FLINK-9290) The job is unable to recover from a checkpoint

[ https://issues.apache.org/jira/browse/FLINK-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461167#comment-16461167 ] Stefan Richter commented on FLINK-9290: --- [~sihuazhou] yes, looks like it. > The job is una

[jira] [Commented] (FLINK-7484) CaseClassSerializer.duplicate() does not perform proper deep copy

[ https://issues.apache.org/jira/browse/FLINK-7484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460907#comment-16460907 ] Stefan Richter commented on FLINK-7484: --- [~joshlemer] this looks like a different problem to me

[jira] [Closed] (FLINK-9263) Kafka010ITCase failed on travis because of the concurrency problem in DefaultOperateStateBackend

[ https://issues.apache.org/jira/browse/FLINK-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-9263. - Resolution: Fixed Merged in: master: c11f11359b release 1.5: de4f283087 > Kafka010ITC

[jira] [Closed] (FLINK-9270) Upgrade RocksDB to 5.11.3, and resolve concurrent test invocation problem of @RetryOnFailure

[ https://issues.apache.org/jira/browse/FLINK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-9270. - Resolution: Won't Fix Won't fix until we figure out a way around the performance regression

[jira] [Commented] (FLINK-9268) RockDB errors from WindowOperator

[ https://issues.apache.org/jira/browse/FLINK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457505#comment-16457505 ] Stefan Richter commented on FLINK-9268: --- This is a known issue with RocksDB, see [https

[jira] [Closed] (FLINK-8992) Implement source and operator that validate exactly-once

[ https://issues.apache.org/jira/browse/FLINK-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8992. - Resolution: Fixed Merged in: master: 31c717697a52af2c64439aaaba2f6a1f97a22159 release 1.5

Re: Using RocksDB as State Backend over a Distributed File System

roidUsers_wl=ym_sub1=Internal_sub2=Global_YGrowth_sub3=EmailSignature> > On Thu, 26 Apr 2018 at 5:30 PM, Stefan Richter > <s.rich...@data-artisans.com> wrote: > Hi, > > I think there is a misunderstanding. RocksDB state backend always operates on > local disk

Re: Using RocksDB as State Backend over a Distributed File System

hirag > > Sent from Yahoo Mail on Android > <https://go.onelink.me/107872968?pid=InProduct=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers_wl=ym_sub1=Internal_sub2=Global_YGrowth_sub3=EmailSignature> > On Thu, 26 Apr 2018 at 5:30 PM, Stefan Richter > <s.rich...@data-a

Re: Using RocksDB as State Backend over a Distributed File System

Hi, I think there is a misunderstanding. RocksDB state backend always operates on local disk of the node that runs your task to give you optimal performance. You can think of this as a transient working area that does not require any durability. Durability always happens through checkpoints

[jira] [Closed] (FLINK-5372) Fix RocksDBAsyncSnapshotTest.testCancelFullyAsyncCheckpoints()

[ https://issues.apache.org/jira/browse/FLINK-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-5372. - Resolution: Fixed Fix Version/s: (was: 1.4.3) 1.6.0 Fixed

[jira] [Closed] (FLINK-8836) Duplicating a KryoSerializer does not duplicate registered default serializers

[ https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8836. - Resolution: Fixed Fix Version/s: 1.4.0 1.6.0 Merged in: master

[jira] [Commented] (FLINK-9080) Flink Scheduler goes OOM, suspecting a memory leak

[ https://issues.apache.org/jira/browse/FLINK-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448183#comment-16448183 ] Stefan Richter commented on FLINK-9080: --- [~rohsing] could you somehow provide your heap dump to me

Re: Flink State monitoring

r us it doesn't matter how exact the number is. I would > expect most users to be only interested in monitoring if the total state size > keeps growing (rapidly), or remains about the same. I suppose all of the > options that you suggested would satisfy this need? > > On Fri, Ap

Re: Flink State monitoring

I wonder if comparing the size of a full checkpoint vs total size of an incremental checkpoint to get insights about the keycount is helpful at all because: - Full checkpoints are basically a dump of all key value pairs as written by their serializers, plus their keygoup id. Each key is

Re: Flink State monitoring

Hi, for incremental checkpoints, it is only showing the size of the deltas. It would probably also be possible to report the full size, but the current reporting and UI is only supporting to deliver a single value. In general, some things are rather hard to report. For example, for the heap

[jira] [Assigned] (FLINK-8836) Duplicating a KryoSerializer does not duplicate registered default serializers

[ https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter reassigned FLINK-8836: - Assignee: Stefan Richter > Duplicating a KryoSerializer does not duplicate registe

[jira] [Commented] (FLINK-8836) Duplicating a KryoSerializer does not duplicate registered default serializers

[ https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443973#comment-16443973 ] Stefan Richter commented on FLINK-8836: --- Serialization will probably work in most cases, but still

[jira] [Closed] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`

[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-9022. - Resolution: Fixed Fixed in master: 777cc1ab53 release-1.5: 01f701ab72 > fix resource cl

[jira] [Closed] (FLINK-9144) Spilling batch job hangs

[ https://issues.apache.org/jira/browse/FLINK-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-9144. - Resolution: Fixed Fixed in: master: 388a083c90 release-1.5: f7e621d322 > Spilling batch

[jira] [Reopened] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`

[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter reopened FLINK-9022: --- Assignee: Stefan Richter (was: Sihua Zhou) Reopened because the dispose of backends should

[jira] [Commented] (FLINK-8836) Duplicating a KryoSerializer does not duplicate registered default serializers

[ https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442673#comment-16442673 ] Stefan Richter commented on FLINK-8836: --- That is a very good question, potentially not at all

[jira] [Updated] (FLINK-8836) Duplicating a KryoSerializer does not duplicate registered default serializers

[ https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter updated FLINK-8836: -- Priority: Critical (was: Major) > Duplicating a KryoSerializer does not duplicate registe

[jira] [Commented] (FLINK-8836) Duplicating a KryoSerializer does not duplicate registered default serializers

[ https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442589#comment-16442589 ] Stefan Richter commented on FLINK-8836: --- Hi, I have seen a report on the mailing list that could

[jira] [Comment Edited] (FLINK-8836) Duplicating a KryoSerializer does not duplicate registered default serializers

[ https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442589#comment-16442589 ] Stefan Richter edited comment on FLINK-8836 at 4/18/18 2:29 PM: Hi, I have

[jira] [Closed] (FLINK-9011) YarnResourceManager spamming log file at INFO level

[ https://issues.apache.org/jira/browse/FLINK-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-9011. - Resolution: Fixed Merge in d5ec911f55. > YarnResourceManager spamming log file at INFO le

Re: CaseClassSerializer and/or TraversableSerializer may still not be threadsafe?

Hi, I agree that this looks like a serializer is shared between two threads, one of them being the event processing loop. I am doubting that the problem is with the async fs backend, because there is code in place that will duplicate all serializers for the async snapshot thread and this is

[jira] [Commented] (FLINK-9192) Undo parameterization of StateMachine Example

[ https://issues.apache.org/jira/browse/FLINK-9192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440934#comment-16440934 ] Stefan Richter commented on FLINK-9192: --- +1, I think that the test could build upon the universal

[jira] [Closed] (FLINK-8968) Fix native resource leak caused by ReadOptions

[ https://issues.apache.org/jira/browse/FLINK-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8968. - Resolution: Fixed Merged in f5071d7a61 > Fix native resource leak caused by ReadOpti

[jira] [Closed] (FLINK-8699) Fix concurrency problem in rocksdb full checkpoint

[ https://issues.apache.org/jira/browse/FLINK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8699. - Resolution: Fixed Merged in 21cf59d5ff > Fix concurrency problem in rocksdb full checkpo

Re: Multiple (non-consecutive) keyBy operators in a dataflow

I don’t think there are any particular implications. I would suggest to go for a simple keyBy and think about optimization if there should actually be a problem at hand. Best, Stefan > Am 03.04.2018 um 17:08 schrieb Timo Walther : > > @Richter: Are you aware of any per-key

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401933#comment-16401933 ] Stefan Richter commented on FLINK-8922: --- Hmm, I wonder why that should be a fix? Does that mean

[jira] [Created] (FLINK-9003) Add operators with input type that goes through custom, stateful serialization

Stefan Richter created FLINK-9003: - Summary: Add operators with input type that goes through custom, stateful serialization Key: FLINK-9003 URL: https://issues.apache.org/jira/browse/FLINK-9003

[jira] [Created] (FLINK-9003) Add operators with input type that goes through custom, stateful serialization

Stefan Richter created FLINK-9003: - Summary: Add operators with input type that goes through custom, stateful serialization Key: FLINK-9003 URL: https://issues.apache.org/jira/browse/FLINK-9003

[jira] [Created] (FLINK-9002) Add operators with input type that goes through Avro serialization (with schema/generic)

Stefan Richter created FLINK-9002: - Summary: Add operators with input type that goes through Avro serialization (with schema/generic) Key: FLINK-9002 URL: https://issues.apache.org/jira/browse/FLINK-9002

[jira] [Created] (FLINK-9002) Add operators with input type that goes through Avro serialization (with schema/generic)

Stefan Richter created FLINK-9002: - Summary: Add operators with input type that goes through Avro serialization (with schema/generic) Key: FLINK-9002 URL: https://issues.apache.org/jira/browse/FLINK-9002

[jira] [Created] (FLINK-9001) Add operators with input type that goes through Kryo serialization (registered/generic/custom)

Stefan Richter created FLINK-9001: - Summary: Add operators with input type that goes through Kryo serialization (registered/generic/custom) Key: FLINK-9001 URL: https://issues.apache.org/jira/browse/FLINK-9001

[jira] [Created] (FLINK-9001) Add operators with input type that goes through Kryo serialization (registered/generic/custom)

Stefan Richter created FLINK-9001: - Summary: Add operators with input type that goes through Kryo serialization (registered/generic/custom) Key: FLINK-9001 URL: https://issues.apache.org/jira/browse/FLINK-9001

[jira] [Created] (FLINK-8999) Ensure the job has an operator with operator state.

Stefan Richter created FLINK-8999: - Summary: Ensure the job has an operator with operator state. Key: FLINK-8999 URL: https://issues.apache.org/jira/browse/FLINK-8999 Project: Flink Issue

[jira] [Created] (FLINK-8999) Ensure the job has an operator with operator state.

Stefan Richter created FLINK-8999: - Summary: Ensure the job has an operator with operator state. Key: FLINK-8999 URL: https://issues.apache.org/jira/browse/FLINK-8999 Project: Flink Issue

[jira] [Updated] (FLINK-8998) Add user-defined watermark assigner and timestamp extractor to the test job

[ https://issues.apache.org/jira/browse/FLINK-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter updated FLINK-8998: -- Component/s: Tests > Add user-defined watermark assigner and timestamp extractor to the t

[jira] [Updated] (FLINK-8998) Add user-defined watermark assigner and timestamp extractor to the test job

[ https://issues.apache.org/jira/browse/FLINK-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter updated FLINK-8998: -- Affects Version/s: 1.5.0 > Add user-defined watermark assigner and timestamp extrac

[jira] [Created] (FLINK-8998) Add user-defined watermark assigner and timestamp extractor to the test job

Stefan Richter created FLINK-8998: - Summary: Add user-defined watermark assigner and timestamp extractor to the test job Key: FLINK-8998 URL: https://issues.apache.org/jira/browse/FLINK-8998 Project

[jira] [Created] (FLINK-8998) Add user-defined watermark assigner and timestamp extractor to the test job

Stefan Richter created FLINK-8998: - Summary: Add user-defined watermark assigner and timestamp extractor to the test job Key: FLINK-8998 URL: https://issues.apache.org/jira/browse/FLINK-8998 Project

[jira] [Created] (FLINK-8997) Add sliding window aggregation to the job

Stefan Richter created FLINK-8997: - Summary: Add sliding window aggregation to the job Key: FLINK-8997 URL: https://issues.apache.org/jira/browse/FLINK-8997 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-8997) Add sliding window aggregation to the job

Stefan Richter created FLINK-8997: - Summary: Add sliding window aggregation to the job Key: FLINK-8997 URL: https://issues.apache.org/jira/browse/FLINK-8997 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-8996) Include an operator with broadcast and union state

Stefan Richter created FLINK-8996: - Summary: Include an operator with broadcast and union state Key: FLINK-8996 URL: https://issues.apache.org/jira/browse/FLINK-8996 Project: Flink Issue

[jira] [Created] (FLINK-8996) Include an operator with broadcast and union state

Stefan Richter created FLINK-8996: - Summary: Include an operator with broadcast and union state Key: FLINK-8996 URL: https://issues.apache.org/jira/browse/FLINK-8996 Project: Flink Issue

[jira] [Created] (FLINK-8995) Add a test operator with keyed state that uses custom, stateful serializer

Stefan Richter created FLINK-8995: - Summary: Add a test operator with keyed state that uses custom, stateful serializer Key: FLINK-8995 URL: https://issues.apache.org/jira/browse/FLINK-8995 Project

[jira] [Created] (FLINK-8995) Add a test operator with keyed state that uses custom, stateful serializer

Stefan Richter created FLINK-8995: - Summary: Add a test operator with keyed state that uses custom, stateful serializer Key: FLINK-8995 URL: https://issues.apache.org/jira/browse/FLINK-8995 Project

[jira] [Updated] (FLINK-8993) Add a test operator with keyed state that uses Kryo serializer (registered/unregistered/custom)

[ https://issues.apache.org/jira/browse/FLINK-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter updated FLINK-8993: -- Description: Add an operator with keyed state that uses Kryo serializer (registered

[jira] [Created] (FLINK-8994) Add a test operator with keyed state that uses Avro serializer (from schema/by reflection)

Stefan Richter created FLINK-8994: - Summary: Add a test operator with keyed state that uses Avro serializer (from schema/by reflection) Key: FLINK-8994 URL: https://issues.apache.org/jira/browse/FLINK-8994

[jira] [Updated] (FLINK-8993) Add a test operator with keyed state that uses Kryo serializer (registered/unregistered/custom)

[ https://issues.apache.org/jira/browse/FLINK-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter updated FLINK-8993: -- Summary: Add a test operator with keyed state that uses Kryo serializer (registered

[jira] [Created] (FLINK-8994) Add a test operator with keyed state that uses Avro serializer (from schema/by reflection)

Stefan Richter created FLINK-8994: - Summary: Add a test operator with keyed state that uses Avro serializer (from schema/by reflection) Key: FLINK-8994 URL: https://issues.apache.org/jira/browse/FLINK-8994

[jira] [Created] (FLINK-8993) Add a test operator with keyed state that uses Kryo serializer (registered/unregistered)

Stefan Richter created FLINK-8993: - Summary: Add a test operator with keyed state that uses Kryo serializer (registered/unregistered) Key: FLINK-8993 URL: https://issues.apache.org/jira/browse/FLINK-8993

[jira] [Created] (FLINK-8993) Add a test operator with keyed state that uses Kryo serializer (registered/unregistered)

Stefan Richter created FLINK-8993: - Summary: Add a test operator with keyed state that uses Kryo serializer (registered/unregistered) Key: FLINK-8993 URL: https://issues.apache.org/jira/browse/FLINK-8993

[jira] [Created] (FLINK-8992) Implement source and operator that validate exactly-once

Stefan Richter created FLINK-8992: - Summary: Implement source and operator that validate exactly-once Key: FLINK-8992 URL: https://issues.apache.org/jira/browse/FLINK-8992 Project: Flink

[jira] [Created] (FLINK-8992) Implement source and operator that validate exactly-once

Stefan Richter created FLINK-8992: - Summary: Implement source and operator that validate exactly-once Key: FLINK-8992 URL: https://issues.apache.org/jira/browse/FLINK-8992 Project: Flink

Re: Slow flink checkpoint

Hi, yes, that is correct, the timer service is currently only available in main-memory and only with synchronous snapshots. this topic is on our TODO list for after the Flink 1.5 release. Best, Stefan > Am 16.03.2018 um 09:03 schrieb Fabian Hueske : > > Hi, > > AFAIK,

[jira] [Closed] (FLINK-8927) Eagerly release the checkpoint object created from RocksDB

[ https://issues.apache.org/jira/browse/FLINK-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8927. - Resolution: Fixed Merged in 3debf47e5d. > Eagerly release the checkpoint object created f

[jira] [Resolved] (FLINK-8913) RocksDB state backend crashes in alpine image

[ https://issues.apache.org/jira/browse/FLINK-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter resolved FLINK-8913. --- Resolution: Won't Fix Release Note: RocksDB requires glibc which is not available

[jira] [Commented] (FLINK-8913) RocksDB state backend crashes in alpine image

[ https://issues.apache.org/jira/browse/FLINK-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395360#comment-16395360 ] Stefan Richter commented on FLINK-8913: --- This is a known issue, Alpine Linux uses MUSL instead

[jira] [Closed] (FLINK-8042) Retry individual failover-strategy for some time first before reverting to full job restart

[ https://issues.apache.org/jira/browse/FLINK-8042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8042. - Resolution: Won't Fix This problem should be solved with FLIP-6 and is hard to support

[jira] [Commented] (FLINK-8042) Retry individual failover-strategy for some time first before reverting to full job restart

[ https://issues.apache.org/jira/browse/FLINK-8042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395297#comment-16395297 ] Stefan Richter commented on FLINK-8042: --- [~till.rohrmann] and me came to the conclusion

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395205#comment-16395205 ] Stefan Richter commented on FLINK-8922: --- Yes, I tried out to create a dedicated \{{WriteOptions

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395054#comment-16395054 ] Stefan Richter commented on FLINK-8922: --- Ok that sounds good. I also had a look and it looks strange

[jira] [Resolved] (FLINK-8882) flink-tests fails with core dump

[ https://issues.apache.org/jira/browse/FLINK-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter resolved FLINK-8882. --- Resolution: Fixed > flink-tests fails with core d

[jira] [Assigned] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter reassigned FLINK-8922: - Assignee: Stefan Richter > Revert FLINK-8859 because it causes segfaults in test

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395037#comment-16395037 ] Stefan Richter commented on FLINK-8922: --- FYI [~sihuazhou] > Revert FLINK-8859 because it cau

[jira] [Closed] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8922. - Resolution: Fixed Release Note: Merged in a389b43581. > Revert FLINK-8859 because it cau

[jira] [Created] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

Stefan Richter created FLINK-8922: - Summary: Revert FLINK-8859 because it causes segfaults in testing Key: FLINK-8922 URL: https://issues.apache.org/jira/browse/FLINK-8922 Project: Flink

[jira] [Created] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

Stefan Richter created FLINK-8922: - Summary: Revert FLINK-8859 because it causes segfaults in testing Key: FLINK-8922 URL: https://issues.apache.org/jira/browse/FLINK-8922 Project: Flink

[jira] [Created] (FLINK-8910) Introduce automated end-to-end test for local recovery (including sticky scheduling)

Stefan Richter created FLINK-8910: - Summary: Introduce automated end-to-end test for local recovery (including sticky scheduling) Key: FLINK-8910 URL: https://issues.apache.org/jira/browse/FLINK-8910

[jira] [Created] (FLINK-8910) Introduce automated end-to-end test for local recovery (including sticky scheduling)

Stefan Richter created FLINK-8910: - Summary: Introduce automated end-to-end test for local recovery (including sticky scheduling) Key: FLINK-8910 URL: https://issues.apache.org/jira/browse/FLINK-8910

Re: checkpoint stuck with rocksdb statebackend and s3 filesystem

> Sihua Zhou > > > 发自网易邮箱大师 > > On 03/6/2018 14:45，Tony Wei<tony19920...@gmail.com> > <mailto:tony19920...@gmail.com> wrote： > Sent to the wrong mailing list. Forward it to the correct one. > > -- Forwarded message -- > From: Tony Wei

[jira] [Commented] (FLINK-8882) flink-tests fails with core dump

[ https://issues.apache.org/jira/browse/FLINK-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391314#comment-16391314 ] Stefan Richter commented on FLINK-8882: --- Got to double check again, maybe the problem is also

[jira] [Commented] (FLINK-8882) flink-tests fails with core dump

[ https://issues.apache.org/jira/browse/FLINK-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391300#comment-16391300 ] Stefan Richter commented on FLINK-8882: --- It seems like the problem was introduced by the changes

[jira] [Assigned] (FLINK-8882) flink-tests fails with core dump

[ https://issues.apache.org/jira/browse/FLINK-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter reassigned FLINK-8882: - Assignee: Stefan Richter > flink-tests fails with core d

Re: Table Api and CSV builder

Hi, I think you just need to specify a custom watermark extractor that constructs the watermark from the 3 fields, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_timestamp_extractors.html

[jira] [Commented] (FLINK-8867) Rocksdb checkpointing failing with fs.default-scheme: hdfs:// config

[ https://issues.apache.org/jira/browse/FLINK-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388012#comment-16388012 ] Stefan Richter commented on FLINK-8867: --- The reported exception is not showing the root cause, just

[jira] [Comment Edited] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

[ https://issues.apache.org/jira/browse/FLINK-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387786#comment-16387786 ] Stefan Richter edited comment on FLINK-8871 at 3/6/18 1:40 PM: --- [~sihuazhou

[jira] [Commented] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

[ https://issues.apache.org/jira/browse/FLINK-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387786#comment-16387786 ] Stefan Richter commented on FLINK-8871: --- [~sihuazhou] Given the release plan

Re: checkpoint stuck with rocksdb statebackend and s3 filesystem

Minimum Pause Between Checkpoints 0ms > Maximum Concurrent Checkpoints1 > Persist Checkpoints ExternallyEnabled (delete on cancellation) > Best Regards, > Tony Wei > > 2018-03-05 21:30 GMT+08:00 Stefan Richter <s.rich...@data-artisans.com > <mailto:s

[jira] [Created] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

Stefan Richter created FLINK-8871: - Summary: Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager Key: FLINK-8871 URL: https://issues.apache.org/jira/browse/FLINK-8871

[jira] [Created] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

Stefan Richter created FLINK-8871: - Summary: Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager Key: FLINK-8871 URL: https://issues.apache.org/jira/browse/FLINK-8871

Re: checkpoint stuck with rocksdb statebackend and s3 filesystem

Hi, quick question: what is your exact checkpointing configuration? In particular, what is your value for the maximum parallel checkpoints and the minimum time interval to wait between two checkpoints? Best, Stefan > Am 05.03.2018 um 06:34 schrieb Tony Wei : > > Hi

[jira] [Closed] (FLINK-8859) RocksDB backend should pass WriteOption to Rocks.put() when restoring

[ https://issues.apache.org/jira/browse/FLINK-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8859. - Resolution: Fixed Release Note: Merged in 131daa28bf. > RocksDB backend should p

[jira] [Commented] (FLINK-8837) Move DataStreamUtils to package 'experimental'.

[ https://issues.apache.org/jira/browse/FLINK-8837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383509#comment-16383509 ] Stefan Richter commented on FLINK-8837: --- The class is annotated as evolving and as you said

<    3   4   5   6   7   8   9   10   11   12   >