to do
that: if some “key interval” concept is used, then state can somehow decoupled
from partition number by consistent hashing.
Regards
Jialei
From: Jacek Laskowski
Date: Wednesday, June 26, 2019 at 11:00 AM
To: "Rong, Jialei"
Cc: "user @spark"
Subject: Re: Chang
Hi Dear Spark Expert
I’m curious about a question regarding Spark Streaming/Structured Streaming:
whether it allows to change parallelism number(the default one or the one
specified in particular operator) in a stream having stateful
transform/operator? Whether this will cause my checkpointed
Fantastic, thanks!
From: Jungtaek Lim
Date: Wednesday, June 26, 2019 at 2:59 PM
To: "Rong, Jialei"
Cc: Jacek Laskowski , "user @spark"
Subject: Re: Change parallelism number in Spark Streaming
Hi,
you could consider state operator's partition numbers as "max para
