That sounds fine to me. I’m hoping that this ticket can make it into Spark 2.3: https://issues.apache.org/jira/browse/SPARK-18016 <https://issues.apache.org/jira/browse/SPARK-18016>
It’s causing some pretty considerable problems when we alter the columns to be nullable, but we are OK for now without that. Best, Justin > On Nov 9, 2017, at 4:54 PM, Michael Armbrust <mich...@databricks.com> wrote: > > According to the timeline posted on the website, we are nearing branch cut > for Spark 2.3. I'd like to propose pushing this out towards mid to late > December for a couple of reasons and would like to hear what people think. > > 1. I've done release management during the Thanksgiving / Christmas time > before and in my experience, we don't actually get a lot of testing during > this time due to vacations and other commitments. I think beginning the RC > process in early January would give us the best coverage in the shortest > amount of time. > 2. There are several large initiatives in progress that given a little more > time would leave us with a much more exciting 2.3 release. Specifically, the > work on the history server, Kubernetes and continuous processing. > 3. Given the actual release date of Spark 2.2, I think we'll still get Spark > 2.3 out roughly 6 months after. > > Thoughts? > > Michael
