[jira] [Created] (FLINK-23636) NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted fails on azure

2021-08-04 Thread Xintong Song (Jira)
Xintong Song created FLINK-23636: Summary: NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted fails on azure Key: FLINK-23636 URL: https://issues.apache.org/jira/browse/FLINK-23636 Project:

Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics

2021-08-04 Thread Becket Qin
I am with Thomas on the contextual usage of metrics as well. That is the "global" variable usage I was trying to explain in the discussion thread. Thanks, Jiangjie (Becket) Qin On Thu, Aug 5, 2021 at 1:59 AM Thomas Weise wrote: > I agree with Becket. Although I find the use of gauge to pass

[jira] [Created] (FLINK-23635) PrestoS3FileSystemITCase.testConfigKeysForwarding

2021-08-04 Thread Xintong Song (Jira)
Xintong Song created FLINK-23635: Summary: PrestoS3FileSystemITCase.testConfigKeysForwarding Key: FLINK-23635 URL: https://issues.apache.org/jira/browse/FLINK-23635 Project: Flink Issue

[jira] [Created] (FLINK-23634) BatchingStateChangeUploaderTest.testDelay fails on azure

2021-08-04 Thread Xintong Song (Jira)
Xintong Song created FLINK-23634: Summary: BatchingStateChangeUploaderTest.testDelay fails on azure Key: FLINK-23634 URL: https://issues.apache.org/jira/browse/FLINK-23634 Project: Flink

Re: Looking for Maintainers for Flink on YARN

2021-08-04 Thread Yangze Guo
Thanks Konstantin for bringing this up, and thanks Marton and Gabor for stepping up! Both dates work for me. If there is something our team can help with for the kick-off, please let me know. Best, Yangze Guo On Thu, Aug 5, 2021 at 9:39 AM Xintong Song wrote: > > Yangze will join the kick-off

Re: Looking for Maintainers for Flink on YARN

2021-08-04 Thread Xintong Song
Yangze will join the kick-off on behalf of our team. Thank you~ Xintong Song On Wed, Aug 4, 2021 at 8:36 PM Gabor Somogyi wrote: > Both are good to me. > > G > > > On Wed, Aug 4, 2021 at 9:30 AM Konstantin Knauf wrote: > > > Hi everyone > > > > Thank you Marton & team for stepping up and

[jira] [Created] (FLINK-23633) HybridSource: Support dynamic stop position in FileSource

2021-08-04 Thread Thomas Weise (Jira)
Thomas Weise created FLINK-23633: Summary: HybridSource: Support dynamic stop position in FileSource Key: FLINK-23633 URL: https://issues.apache.org/jira/browse/FLINK-23633 Project: Flink

[jira] [Created] (FLINK-23632) [DOCS]The link to setup-Pyflink-virtual-env.sh is broken for page dev/python/faq

2021-08-04 Thread wuguihu (Jira)
wuguihu created FLINK-23632: --- Summary: [DOCS]The link to setup-Pyflink-virtual-env.sh is broken for page dev/python/faq Key: FLINK-23632 URL: https://issues.apache.org/jira/browse/FLINK-23632 Project:

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Piotr Nowojski
Thanks for the detailed explanation Yun Tang and clearly all of the effort you have put into it. Based on what was described here I would also vote for going forward with the upgrade. It's a pity that this regression wasn't caught in the RocksDB community. I would have two questions/ideas: 1. Can

Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics

2021-08-04 Thread Thomas Weise
I agree with Becket. Although I find the use of gauge to pass contextual information less intuitive, it is acceptable within the metric group interface (plus javadoc). Thanks, Thomas On Tue, Aug 3, 2021 at 10:06 PM Becket Qin wrote: > Personally speaking, it is intuitive for me to set a gauge

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Yun Tang
Hi Yuval, Upgrading RocksDB version is a long story since Flink-1.10. When we first plan to introduce write buffer manager to help control the memory usage of RocksDB, we actually wanted to bump up to RocksDB-5.18 from current RocksDB-5.17. However, we found performance regression in our micro

[jira] [Created] (FLINK-23631) [docs-zh] The name attribute of GH_link in shortcodes was lost. As a result, the Link to Flink-examexam-batch failed

2021-08-04 Thread wuguihu (Jira)
wuguihu created FLINK-23631: --- Summary: [docs-zh] The name attribute of GH_link in shortcodes was lost. As a result, the Link to Flink-examexam-batch failed Key: FLINK-23631 URL:

[jira] [Created] (FLINK-23630) Make EventTimeWindowCheckpointingITCase and LocalRecoveryITCase run on Windows.

2021-08-04 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23630: Summary: Make EventTimeWindowCheckpointingITCase and LocalRecoveryITCase run on Windows. Key: FLINK-23630 URL: https://issues.apache.org/jira/browse/FLINK-23630

[jira] [Created] (FLINK-23629) Remove redundant test cases in EventTimeWindowCheckpointingITCase

2021-08-04 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-23629: Summary: Remove redundant test cases in EventTimeWindowCheckpointingITCase Key: FLINK-23629 URL: https://issues.apache.org/jira/browse/FLINK-23629 Project: Flink

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Nico Kruber
That's actually also what I'm seeing most of the time and what I'd expect to improve with the newer RocksDB version. Hence, I'd also favour the upgrade even if there is a slight catch with respect to performance - we should, however, continue to investigate this together with the RocksDB

[jira] [Created] (FLINK-23628) MeasurementInfoProviderTest fails on Azure

2021-08-04 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-23628: Summary: MeasurementInfoProviderTest fails on Azure Key: FLINK-23628 URL: https://issues.apache.org/jira/browse/FLINK-23628 Project: Flink Issue

[jira] [Created] (FLINK-23627) Migrate all OperatorMG instantiations to factory method

2021-08-04 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23627: Summary: Migrate all OperatorMG instantiations to factory method Key: FLINK-23627 URL: https://issues.apache.org/jira/browse/FLINK-23627 Project: Flink

[jira] [Created] (FLINK-23626) Migrate all TaskMG instantiations to factory method

2021-08-04 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23626: Summary: Migrate all TaskMG instantiations to factory method Key: FLINK-23626 URL: https://issues.apache.org/jira/browse/FLINK-23626 Project: Flink

[jira] [Created] (FLINK-23625) Migrate all TaskManagerJobMG instantiations to factory method

2021-08-04 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23625: Summary: Migrate all TaskManagerJobMG instantiations to factory method Key: FLINK-23625 URL: https://issues.apache.org/jira/browse/FLINK-23625 Project: Flink

[jira] [Created] (FLINK-23624) Migrate all TaskManagerMG instantiations to factory method

2021-08-04 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23624: Summary: Migrate all TaskManagerMG instantiations to factory method Key: FLINK-23624 URL: https://issues.apache.org/jira/browse/FLINK-23624 Project: Flink

[jira] [Created] (FLINK-23623) Over expression should allow to order by any fields in batch mode

2021-08-04 Thread Shengkai Fang (Jira)
Shengkai Fang created FLINK-23623: - Summary: Over expression should allow to order by any fields in batch mode Key: FLINK-23623 URL: https://issues.apache.org/jira/browse/FLINK-23623 Project: Flink

[jira] [Created] (FLINK-23622) Default filesystem scheme improvement

2021-08-04 Thread zlzhang0122 (Jira)
zlzhang0122 created FLINK-23622: --- Summary: Default filesystem scheme improvement Key: FLINK-23622 URL: https://issues.apache.org/jira/browse/FLINK-23622 Project: Flink Issue Type: Improvement

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Stephan Ewen
For the details of what causes this regression, I would add @Yun Tang to this discussion. On Wed, Aug 4, 2021 at 2:36 PM Yuval Itzchakov wrote: > We are heavy users of RocksDB and have had several issues with memory > managed in Kubernetes, most of them actually went away when we upgraded >

Re: Looking for Maintainers for Flink on YARN

2021-08-04 Thread Gabor Somogyi
Both are good to me. G On Wed, Aug 4, 2021 at 9:30 AM Konstantin Knauf wrote: > Hi everyone > > Thank you Marton & team for stepping up and thanks, Xintong, for offering > your continued support. I'd like to leave this thread "open" for a bit > longer so that others can continue to chime in.

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread David Anderson
I am hearing quite often from users who are struggling to manage memory usage, and these are all users using RocksDB. While I don't know for certain that RocksDB is the cause in every case, from my perspective, getting the better memory stability of version 6.20 in place is critical. Regards,

[jira] [Created] (FLINK-23621) FLIP-177: Extend Sink API

2021-08-04 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-23621: --- Summary: FLIP-177: Extend Sink API Key: FLINK-23621 URL: https://issues.apache.org/jira/browse/FLINK-23621 Project: Flink Issue Type: Improvement

[ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Stephan Ewen
Hi all! *!!! If you are a big user of the Embedded RocksDB State Backend and have performance sensitive workloads, please read this !!!* I want to quickly raise some awareness for a RocksDB version upgrade we plan to do, and some possible impact on application performance. *We plan to upgrade

[jira] [Created] (FLINK-23620) Introduce proper YAML parsing to Flink's configuration

2021-08-04 Thread Mika Naylor (Jira)
Mika Naylor created FLINK-23620: --- Summary: Introduce proper YAML parsing to Flink's configuration Key: FLINK-23620 URL: https://issues.apache.org/jira/browse/FLINK-23620 Project: Flink Issue

[RESULT] [VOTE] FLIP-177: Extend Sink API

2021-08-04 Thread Arvid Heise
Dear devs, I'm happy to announce that we have unanimously approved this FLIP. There are 3 approving votes of which 3 are binding: Thomas Weise (binding) Danny Cranmer (binding) Guowei Ma (binding) Best, Arvid On Mon, Aug 2, 2021 at 11:10 AM Guowei Ma wrote: > +1(binding) > Best, > Guowei >

[RESULT] [VOTE] FLIP-179: Expose Standardized Operator Metrics

2021-08-04 Thread Arvid Heise
Dear devs, I'm happy to announce that we have unanimously approved this FLIP. There are 4 approving votes of which 3 are binding: Steven Wu (non-binding) Jiangjie (Becket) Qin (binding) Chesnay Schepler (binding) Thomas Weise (binding) Best, Arvid On Wed, Aug 4, 2021 at 7:06 AM Becket Qin

[jira] [Created] (FLINK-23619) Remove PythonTimestampsAndWatermarksOperator

2021-08-04 Thread Dian Fu (Jira)
Dian Fu created FLINK-23619: --- Summary: Remove PythonTimestampsAndWatermarksOperator Key: FLINK-23619 URL: https://issues.apache.org/jira/browse/FLINK-23619 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23618) Commpilation error for MeasurementInfoProviderTest

2021-08-04 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-23618: Summary: Commpilation error for MeasurementInfoProviderTest Key: FLINK-23618 URL: https://issues.apache.org/jira/browse/FLINK-23618 Project: Flink

[jira] [Created] (FLINK-23617) Co-locate sink operators with same parallelism

2021-08-04 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-23617: --- Summary: Co-locate sink operators with same parallelism Key: FLINK-23617 URL: https://issues.apache.org/jira/browse/FLINK-23617 Project: Flink Issue Type:

[jira] [Created] (FLINK-23616) Support to chain for the Python operators as much as possible

2021-08-04 Thread Dian Fu (Jira)
Dian Fu created FLINK-23616: --- Summary: Support to chain for the Python operators as much as possible Key: FLINK-23616 URL: https://issues.apache.org/jira/browse/FLINK-23616 Project: Flink Issue

Re: Looking for Maintainers for Flink on YARN

2021-08-04 Thread Konstantin Knauf
Hi everyone Thank you Marton & team for stepping up and thanks, Xintong, for offering your continued support. I'd like to leave this thread "open" for a bit longer so that others can continue to chime in. In the meantime, I would already like to propose two dates for an informal kick-off: * Thu,

[jira] [Created] (FLINK-23615) Documentation of the TRUNCATE SQL function is not correct

2021-08-04 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-23615: --- Summary: Documentation of the TRUNCATE SQL function is not correct Key: FLINK-23615 URL: https://issues.apache.org/jira/browse/FLINK-23615 Project: Flink

[jira] [Created] (FLINK-23614) The resulting scale of TRUNCATE(DECIMAL, ...) is not correct

2021-08-04 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-23614: --- Summary: The resulting scale of TRUNCATE(DECIMAL, ...) is not correct Key: FLINK-23614 URL: https://issues.apache.org/jira/browse/FLINK-23614 Project: Flink

Re: Unable to read state Witten by Beam application with Flink runner using Flink's State Processor API

2021-08-04 Thread David Morávek
Hi Sandeep, thanks for the example, I'll take a look into it and will get back to you ;) On Tue, Aug 3, 2021 at 9:44 PM Kathula, Sandeep wrote: > Hi David, > Thanks for the reply. I tried with Beam 2.29 and Flink > 1.12 and still getting NullPointerException like before. I