Hi Padarn, After switch stateBackend from filesystem to rocksdb, all reads/writes from/to backend have to go through de-/serialization to retrieve/store the state objects, this may cause more cpu cost. But I'm not sure it is the main reason leads to 3x CPU cost in your job. To find out the reason, we need more profile on CPU cost, such as Flame Graphs. BTW, starting with Flink 1.13, Flame Graphs are natively supported in Flink[1].
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/debugging/flame_graphs/ Best, JING ZHANG Padarn Wilson <pad...@gmail.com> 于2021年6月15日周二 下午5:05写道： > Hi all, > > We have a job that we just enabled rocksdb on (instead of file backend), > and see that the CPU usage is almost 3x greater on (we had to increase > taskmanagers 3x to get it to run. > > I don't really understand this, is there something we can look at to > understand why CPU use is so high? Our state mostly consists of aggregation > windows. > > Cheers, > Padarn >
