from:"cs"

目前Kafka producer的partitioner使用的是org.apache.flink.streaming.connectors.kafka.partitioner.FlinkFixedPartitioner这个类具体的分区方法是org.apache.flink.streaming.connectors.kafka.partitioner.FlinkFixedPartitioner#partition 由该方法可以得知你每个subtask发到哪个kafka的partition中。每个subtask的数据只会写到一个固定的partition里面。

?????? Flink SQL??????????????????????(??????)????????????

2020-07-08 文章 cs

join ---- ??:"godfrey he"

?????? ??????????

2020-06-28 文章 cs

??yarnyarn?? yarn??standalone??standalone??flink??, yarnspark??MR?? ---- ??:"LakeShen"

???????????????????????????? operator

2020-06-24 文章 cs

1. ??operatordisableChainingoperator?? 2. tm ---- ??:""

?????? flink???????????????? flink last checkpoint ???????????????????????????? state ??????????????

2020-06-24 文章 cs

??checkpoint?? StreamExecutionEnvironment.getCheckpointConfig().setFailOnCheckpointingErrors(true); /** * Sets the expected behaviour for tasks in case that they encounter an error in their checkpointing procedure. * If this is set to true, the task will fail

Streaming File Sink??????????

2020-03-16 文章 cs

Streaming File Sinkparquet avrobulk writefinal StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); env.setParallelism(1); env.enableCheckpointing(60 * 1000, CheckpointingMode.AT_LEAST_ONCE); env.setStateBackend(new FsStateBackend(new

?????? Re: flink??????????????

2020-01-01 文章 cs

??tm??gc??flink1.8 heap10.25GB??heap??1/33.41GB ??heap2442764288byte ??jvm ---- ??:"Xintong Song"

?????? flink??????????????

2019-12-31 文章 cs

?? ??tm-XX:NewSize tm15G -XX:NewSize=2442764288 tm20G ?? -XX:NewSize=2442764288 ?? ---- ??:"Xintong Song"

flink??????????????

2019-12-31 文章 cs

taskmanager15Gheap10G tm??1.cutoff(15GB * 0.25) 2.heap(heap15GB - cutoff) 3.offheap(offheap??15GB-heap) offheap??-XX:MaxDirectMemorySize?? MaxDirectMemorySize??