[ https://issues.apache.org/jira/browse/SPARK-23796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-23796. ---------------------------------- Resolution: Incomplete > There's no API to change state RDD's name > ----------------------------------------- > > Key: SPARK-23796 > URL: https://issues.apache.org/jira/browse/SPARK-23796 > Project: Spark > Issue Type: Question > Components: Spark Core > Affects Versions: 2.3.0 > Reporter: István Gansperger > Priority: Minor > Labels: bulk-closed > > I use a few {{mapWithState}} stream oparations in my application and at some > point it became a minor inconvenience that I could not figure out how to set > the state RDDs name or serialization level. Searching around didn't really > help and I have not come across any issues regarding this (pardon my > inability to find it if there's one). It could be useful to see how much > memory each state uses if the user has multiple such transformations. > I have used some ugly reflection based code to be able to set the name of the > state RDD and also the serialization level. I understand that the latter may > be intentionally limited, but I haven't come across any issues caused by this > apart from slightly degraded performance in exchange for a bit less memory > usage. Are these limitations in place intentionally or is it just an > oversight? Having some extra methods for these on {{StateSpec}} could be > useful in my opinion. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org