If you are using capacity scheduler in yarn: By default yarn capacity
scheduler uses DefaultResourceCalculator. DefaultResourceCalculator
considerĀ¹s only memory while allocating contains.
You can use DominantResourceCalculator, it considers memory and cpu.
In capacity-scheduler.xml set
HiCan we share spark streaming state between two DStreams??Basically I want to
create state using first stream and enrich second stream using state.Example: I
have modified StatefulNetworkWordCount example. I am creating state using first
stream and enriching second stream with count of first
Hi
What is the right way of using spark2.0 state store feature in spark
streaming??I referred test cases in
this(https://github.com/apache/spark/pull/11645/files) pull request and
implemented word count using state store.My source is kafka(1 topic, 10
partitions). My data pump is pushing