Flink memory management in table api

2018-07-03 Thread Amol S - iProgrammer
Hello folks, I am using flink table api to join multiple tables and create a single table from them. I have some doubts in my mind. 1. How long the query will maintain partial results per key and how it maintains state of each key? 2. If it is maintains state in memory then the memory will conti

Re: Flink memory management in table api

2018-07-04 Thread Fabian Hueske
Hi Amol, The memory consumption depends on the query/operation that you are doing. Time-based operations like group-window-aggregations, over-window-aggregations, or window-joins can automatically clean up their state once data is not no longer needed. Operations such as non-windowed aggregations

Re: Flink memory management in table api

2018-07-04 Thread Amol S - iProgrammer
Hello fabian, Thanks for your quick response, According to above conversation flink will persist state forever for non windowed operations. I want to know how flink persiat the state i.e. Database or file system or in memory etc. On Wed, 4 Jul 2018 at 2:12 PM, Fabian Hueske wrote: > Hi Amol, >

Re: Flink memory management in table api

2018-07-04 Thread Fabian Hueske
State is maintained in the configured state backend [1]. Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/state_backends.html 2018-07-04 11:38 GMT+02:00 Amol S - iProgrammer : > Hello fabian, > > Thanks for your quick response, > > According to above conver