Re: Resource Optimization for Flink Job in AWS EMR Cluster

2020-11-04 Thread Satyaa Dixit
Hi Deep, Thanks for bringing this on table, I'm also facing a similar kind of issue while deploying my flink Job w.r.t resources optimization. Hi Team, It would be much appreciated if someone helps us here. Regards, Satya On Wed, Nov 4, 2020 at 6:33 PM DEEP NARAYAN Singh wrote: > Hi All,

[jira] [Created] (FLINK-19990) MultipleInputNodeCreationProcessor#isChainableSource should consider DataStreamScanProvider

2020-11-04 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-19990: --- Summary: MultipleInputNodeCreationProcessor#isChainableSource should consider DataStreamScanProvider Key: FLINK-19990 URL: https://issues.apache.org/jira/browse/FLINK-19990

[jira] [Created] (FLINK-19989) Add collect operation in Python DataStream API

2020-11-04 Thread Dian Fu (Jira)
Dian Fu created FLINK-19989: --- Summary: Add collect operation in Python DataStream API Key: FLINK-19989 URL: https://issues.apache.org/jira/browse/FLINK-19989 Project: Flink Issue Type: Improvement

Re: Resource Optimization for Flink Job in AWS EMR Cluster

2020-11-04 Thread DEEP NARAYAN Singh
Hi Guys, Sorry to bother you again.Someone could help me here for clarifying my doubt. Any help will be highly appreciated. Thanks, -Deep On Wed, Nov 4, 2020 at 6:26 PM DEEP NARAYAN Singh wrote: > Hi All, > > I am running a flink streaming job in EMR Cluster with parallelism 21 > having 500 rec

[jira] [Created] (FLINK-19988) Make Approximate Local Recovery Compatible With Unaligned Checkpoint

2020-11-04 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-19988: Summary: Make Approximate Local Recovery Compatible With Unaligned Checkpoint Key: FLINK-19988 URL: https://issues.apache.org/jira/browse/FLINK-19988 Project: Flink

[jira] [Created] (FLINK-19987) HBaseDynamicTableFactoryTest.testTableSourceFactory failed with "NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Obje

2020-11-04 Thread Dian Fu (Jira)
Dian Fu created FLINK-19987: --- Summary: HBaseDynamicTableFactoryTest.testTableSourceFactory failed with "NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V" Key: FLINK-19987

[jira] [Created] (FLINK-19986) The automatic license check failed

2020-11-04 Thread Dian Fu (Jira)
Dian Fu created FLINK-19986: --- Summary: The automatic license check failed Key: FLINK-19986 URL: https://issues.apache.org/jira/browse/FLINK-19986 Project: Flink Issue Type: Bug Components

[jira] [Created] (FLINK-19985) job went into zombie state after ZK session timeout

2020-11-04 Thread Lu Niu (Jira)
Lu Niu created FLINK-19985: -- Summary: job went into zombie state after ZK session timeout Key: FLINK-19985 URL: https://issues.apache.org/jira/browse/FLINK-19985 Project: Flink Issue Type: Bug A

[jira] [Created] (FLINK-19984) Add TypeSerializerTestCoverageTest to check whether tests based on SerializerTestBase and TypeSerializerUpgradeTestBase

2020-11-04 Thread Nicholas Jiang (Jira)
Nicholas Jiang created FLINK-19984: -- Summary: Add TypeSerializerTestCoverageTest to check whether tests based on SerializerTestBase and TypeSerializerUpgradeTestBase Key: FLINK-19984 URL: https://issues.apache.or

Re: [VOTE] Apache Flink Stateful Functions 2.2.1, release candidate #1

2020-11-04 Thread Robert Metzger
+1 binding Checks: - source sha and signature - mvn clean install passed on source archive - staging artifacts have correct version, also in quickstart, NOTICE files seem correct On Wed, Nov 4, 2020 at 11:20 AM Tzu-Li (Gordon) Tai wrote: > Hi everyone, > > Please review and vote on the releas

[DISCUSS] Planning the 1.12 release testing

2020-11-04 Thread Robert Metzger
Hi all, We are soon approaching the feature freeze of the 1.12 release. I would like to make the stabilization phase as short as possible, so that we can release in a timely manner. One important aspect of making the stabilization phase short is knowing all the major bugs. For testing Flink in a

[jira] [Created] (FLINK-19983) ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle unstable

2020-11-04 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19983: -- Summary: ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle unstable Key: FLINK-19983 URL: https://issues.apache.org/jira/browse/FLINK-19983

[jira] [Created] (FLINK-19982) AggregateReduceGroupingITCase.testSingleAggOnTable_SortAgg

2020-11-04 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19982: -- Summary: AggregateReduceGroupingITCase.testSingleAggOnTable_SortAgg Key: FLINK-19982 URL: https://issues.apache.org/jira/browse/FLINK-19982 Project: Flink

[jira] [Created] (FLINK-19981) Improve dealing with Row in DataStream API

2020-11-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-19981: Summary: Improve dealing with Row in DataStream API Key: FLINK-19981 URL: https://issues.apache.org/jira/browse/FLINK-19981 Project: Flink Issue Type: Sub-ta

[jira] [Created] (FLINK-19980) Add StreamTableEnvironment.to/fromChangelogStream

2020-11-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-19980: Summary: Add StreamTableEnvironment.to/fromChangelogStream Key: FLINK-19980 URL: https://issues.apache.org/jira/browse/FLINK-19980 Project: Flink Issue Type:

[jira] [Created] (FLINK-19979) Sanity check after bash e2e tests for no leftover processes

2020-11-04 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19979: -- Summary: Sanity check after bash e2e tests for no leftover processes Key: FLINK-19979 URL: https://issues.apache.org/jira/browse/FLINK-19979 Project: Flink

[jira] [Created] (FLINK-19978) Add StreamStatementSet.attachToStream

2020-11-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-19978: Summary: Add StreamStatementSet.attachToStream Key: FLINK-19978 URL: https://issues.apache.org/jira/browse/FLINK-19978 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-19977) Add new toDataStream and toDataStream

2020-11-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-19977: Summary: Add new toDataStream and toDataStream Key: FLINK-19977 URL: https://issues.apache.org/jira/browse/FLINK-19977 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-19976) FLIP-136: Improve interoperability between DataStream and Table API

2020-11-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-19976: Summary: FLIP-136: Improve interoperability between DataStream and Table API Key: FLINK-19976 URL: https://issues.apache.org/jira/browse/FLINK-19976 Project: Flink

[RESULT][VOTE] FLIP-136: Improve interoperability between DataStream and Table API

2020-11-04 Thread Timo Walther
Hi all, sorry, I forgot to close the voting thread for FLIP-136. I will do this now. The voting time for FLIP-136 has passed. - Kurt (binding) - Jingsong (binding) - Jark (binding) - Xingcan (binding) There were 4 votes, 4 of them were binding. There were no disapproving votes. Thus, FLIP-136

[jira] [Created] (FLINK-19975) Bump testconatiners version

2020-11-04 Thread Igal Shilman (Jira)
Igal Shilman created FLINK-19975: Summary: Bump testconatiners version Key: FLINK-19975 URL: https://issues.apache.org/jira/browse/FLINK-19975 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-19974) SQLClientKafkaITCase.testKafka times out while creating topic

2020-11-04 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19974: -- Summary: SQLClientKafkaITCase.testKafka times out while creating topic Key: FLINK-19974 URL: https://issues.apache.org/jira/browse/FLINK-19974 Project: Flink

Resource Optimization for Flink Job in AWS EMR Cluster

2020-11-04 Thread DEEP NARAYAN Singh
Hi All, I am running a flink streaming job in EMR Cluster with parallelism 21 having 500 records per second.But still seeing cpu utilization is approximate 5-8 percent. Below is the long running session command in EMR Cluster having 3 instance of type C52xlarge(8vcore, 16 GB memory, AWS resource)

[jira] [Created] (FLINK-19973) 【Flink-Deployment】YARN CLI Parameter doesn't work

2020-11-04 Thread zhisheng (Jira)
zhisheng created FLINK-19973: Summary: 【Flink-Deployment】YARN CLI Parameter doesn't work Key: FLINK-19973 URL: https://issues.apache.org/jira/browse/FLINK-19973 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-19972) Provide more details when type serializers are not compatible

2020-11-04 Thread Nico Kruber (Jira)
Nico Kruber created FLINK-19972: --- Summary: Provide more details when type serializers are not compatible Key: FLINK-19972 URL: https://issues.apache.org/jira/browse/FLINK-19972 Project: Flink

[jira] [Created] (FLINK-19971) flink-connector-hbase-base depends on hbase-server, instead of hbase-client

2020-11-04 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19971: -- Summary: flink-connector-hbase-base depends on hbase-server, instead of hbase-client Key: FLINK-19971 URL: https://issues.apache.org/jira/browse/FLINK-19971 Proje

[jira] [Created] (FLINK-19970) State leak in CEP Operators (expired events/keys not removed from state)

2020-11-04 Thread Thomas Wozniakowski (Jira)
Thomas Wozniakowski created FLINK-19970: --- Summary: State leak in CEP Operators (expired events/keys not removed from state) Key: FLINK-19970 URL: https://issues.apache.org/jira/browse/FLINK-19970

[jira] [Created] (FLINK-19969) CliFrontendParser does not provide any help for run-application

2020-11-04 Thread Flavio Pompermaier (Jira)
Flavio Pompermaier created FLINK-19969: -- Summary: CliFrontendParser does not provide any help for run-application Key: FLINK-19969 URL: https://issues.apache.org/jira/browse/FLINK-19969 Project:

Re: [DISCUSS] Releasing StateFun hotfix version 2.2.1

2020-11-04 Thread Tzu-Li (Gordon) Tai
Thanks everyone for the feedback. I've just updated the status of Flink 1.11.3 earlier, in its corresponding discussion thread [1]. >From the looks of it, it seems like it makes sense to proceed with StateFun 2.2.1 without waiting for Flink 1.11.3. Since this is also the consensus we've reached h

[VOTE] Apache Flink Stateful Functions 2.2.1, release candidate #1

2020-11-04 Thread Tzu-Li (Gordon) Tai
Hi everyone, Please review and vote on the release candidate #1 for the version 2.2.1 of Apache Flink Stateful Functions, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) ***Testing Guideline*** You can find here [1] a page in the proj

[jira] [Created] (FLINK-19968) Exception in thread "main" java.lang.IllegalStateException: No ExecutorFactory found to execute the application.

2020-11-04 Thread PeteZhang (Jira)
PeteZhang created FLINK-19968: - Summary: Exception in thread "main" java.lang.IllegalStateException: No ExecutorFactory found to execute the application. Key: FLINK-19968 URL: https://issues.apache.org/jira/browse/FL

[jira] [Created] (FLINK-19967) Clean legacy classes for Parquet and Orc

2020-11-04 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-19967: Summary: Clean legacy classes for Parquet and Orc Key: FLINK-19967 URL: https://issues.apache.org/jira/browse/FLINK-19967 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-19966) Filter push down not work for filesystem connector

2020-11-04 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-19966: Summary: Filter push down not work for filesystem connector Key: FLINK-19966 URL: https://issues.apache.org/jira/browse/FLINK-19966 Project: Flink Issue Type

[jira] [Created] (FLINK-19965) Refactor HiveMapredSplitReader to adapt to the new hive source

2020-11-04 Thread Rui Li (Jira)
Rui Li created FLINK-19965: -- Summary: Refactor HiveMapredSplitReader to adapt to the new hive source Key: FLINK-19965 URL: https://issues.apache.org/jira/browse/FLINK-19965 Project: Flink Issue Typ

[jira] [Created] (FLINK-19964) Gelly ITCase stuck on Azure

2020-11-04 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-19964: Summary: Gelly ITCase stuck on Azure Key: FLINK-19964 URL: https://issues.apache.org/jira/browse/FLINK-19964 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-19963) Let the `SinkWriter` support using the `TimerService`

2020-11-04 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-19963: - Summary: Let the `SinkWriter` support using the `TimerService` Key: FLINK-19963 URL: https://issues.apache.org/jira/browse/FLINK-19963 Project: Flink Issue Type: S

[jira] [Created] (FLINK-19962) fix doc: more examples for expanding arrays into a relation

2020-11-04 Thread Mulan (Jira)
Mulan created FLINK-19962: - Summary: fix doc: more examples for expanding arrays into a relation Key: FLINK-19962 URL: https://issues.apache.org/jira/browse/FLINK-19962 Project: Flink Issue Type: Im