[Spark Core] makeRDD() preferredLocations do not appear to be considered

Hi Guys, I asked this in stack overflow here: https://stackoverflow.com/questions/63535720/why-would-preferredlocations-not-be-enforced-on-an-empty-spark-cluster but am hoping there is further help here. I have a 4 node standalone cluster with workers named worker1, worker2 and worker3 and a

arbitrary state handling in python API

Hi, how can I apply arbitrary state handling as provided by the method: mapGroupsWithState in the java API from the python side? Currently, it looks like this method is not available on spark 3.x in the structured streaming python API. Best, Georg