That's enough. Did you see any error? On Mon, Oct 10, 2016 at 5:08 AM, Joey Echeverria <j...@rocana.com> wrote:
> Hi Ryan! > > Do you know where I need to configure Kryo for this? I already have > spark.serializer=org.apache.spark.serializer.KryoSerializer in my > SparkConf and I registered the class. Is there a different > configuration setting for the state map keys? > > Thanks! > > -Joey > > On Sun, Oct 9, 2016 at 10:58 PM, Shixiong(Ryan) Zhu > <shixi...@databricks.com> wrote: > > You can use Kryo. It also implements KryoSerializable which is supported > by > > Kryo. > > > > On Fri, Oct 7, 2016 at 11:39 AM, Joey Echeverria <j...@rocana.com> > wrote: > >> > >> Looking at the source code for StateMap[1], which is used by > >> JavaPairDStream#mapWithState(), it looks like state keys are > >> serialized using an ObjectOutputStream. I couldn't find a reference to > >> this restriction in the documentation. Did I miss that? > >> > >> Unless I'm mistaken, I'm guessing there isn't a way to use Kryo for > >> this serialization? > >> > >> Thanks! > >> > >> -Joey > >> > >> [1] > >> https://github.com/apache/spark/blob/master/streaming/ > src/main/scala/org/apache/spark/streaming/util/StateMap.scala#L251 > >> > >> --------------------------------------------------------------------- > >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >> > > > > > > -- > -Joey >
