Hi Folks, I just filed https://spark-project.atlassian.net/browse/SPARK-1149 - I'm happy to fix it, but I'd like input about how best to go about it. The problem is, if a partitioner misbehaves by, say, returning a negative partition number, Spark hangs. This is easier to do than it sounds.
I'd like to fix that so that instead we'd get an exception that let the developer know their partitioner had done something wrong. Unfortunately I don't see an easy way to do that without changing method signatures. Would a reasonable compromise be to write a utility method that checks the partition number against Partitioner.numPartitions, and update the code to use that method everywhere that it directly calls Partitioner.getPartition now? Anyone have a better suggestion? Thanks, Bryn