Re:Memory constrains running Flink on Kubernetes

2019-10-10 Thread shengjk1
* 1024 * 1024) .setMaxWriteBufferNumber(1); } } Additional, this is FLINK-7289, it is similar to us. But I don’t find a good way to fix it. [1] https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide [2] https://issues.apache.org/jira/browse/FLINK-7289 Best, Shengjk1 On 07/24

Re: In yarn-cluster model, mapState not support transfer value

2019-09-19 Thread shengjk1
prove performance. Maybe I need to redesign my program. Best, Shengjk1 On 09/19/2019 20:08,Dian Fu wrote: Hi Shengjk1, You should call "mapState.put(key,objectList);" manually after calling "objectList.add(stringObjectTuple2.f1);" to write it to the state backend. Th

In yarn-cluster model, mapState not support transfer value

2019-09-19 Thread shengjk1
keyorder2infos add before objectList.size 0 keyorder2infos add after objectList.size 1 keyorder2infos mapState objectList.size 0 … So, I want to konw : 1.This is a bug or my code has some wrong? 2.Why mapstate on yarn-cluster not support transfer value? I have seen flink source code ,but not find. It runs on: Flink 1.9.0 Java 1.8 Hadoop 2.6.0 Best, Shengjk1

MapState not support transfer value

2019-09-19 Thread shengjk1
()); ... } If I run this in idea the log detail: Best, Shengjk1

Re: Reserving Kafka offset in Flink after modifying app

2019-04-03 Thread shengjk1
Mapbe this page can help you https://ci.apache.org/projects/flink/flink-docs-release-1.7/ops/state/savepoints.html Best, Shengjk1 On 03/26/2019 09:51,Son Mai wrote: Hi Konstantin, Thanks for the response. What still concerned me is: Am I able to recover from checkpoints even if I change

Re:Consume only a few of kafka topic partitions

2019-04-03 Thread shengjk1
Hi, Marke I understand that you want to consume only the specified partition of a topic of kafka. if so, current flink (1.7.x/1.8.x) is not support, you can see https://issues.apache.org/jira/browse/FLINK-11257 Best, Shengjk1 On 03/8/2019 01:03,Marke Builder wrote: Hi

?????? flink savepoints ?? checkpoints??????

2019-03-27 Thread shengjk1
group?? ??flinkkafka consumer??APi assign() kafka subscribe() ??group.id kafka consumer ?? Best, Shengjk1

?????? flink savepoints ?? checkpoints??????

2019-03-26 Thread shengjk1
offsetstate backends??kafka consumer??Api??kafka https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/connectors/kafka.html Best, Shengjk1 ??2019??03??27?? 11:52??<1048095...@qq.com> ?? ??checkpoint??sav

Re:submit job failed on Yarn HA

2019-02-27 Thread shengjk1
tml Best, Shengjk1 On 02/27/2019 15:09,孙森 wrote: Hi all: I run flink (1.5.1 with hadoop 2.7) on yarn ,and submit job by “/usr/local/flink/bin/flink run -m jmhost:port my.jar”, but the submission is failed. The HA configuration is : high-availability: zookeeper high-availability.