Hi all, I've been mostly using Spark with Python, and it's been a great time (thanks for the earlier help with GPUs, btw), but I recently stumbled through the Scala API and found it incredibly rich, with some options that would be pretty helpful for us but are lacking in the Python API. Is it straightforward to write a driver in Scala, but have the workers be written in Python? Alternatively, can I (easily) use Py4J to access these Scala methods from Python? I imagine I'll be playing around with it over the next few days, but I was wondering if anyone had tried this. Sorry if it's a stupid question...
Thanks for the time and attention Patrick Grinaway
