Re: [VOTE] Apache Spark 3.0.0 RC1

Thanks for sharing the blockers, Wenchen. SPARK-31404 has sub-tasks, hence that means all sub-tasks are blockers for this release, do I understand that correctly? Xiao, I sincerely respect the practice the Spark community has been done, so please treat it as 2 cents. Just would like to see the

Re: DSv2 & DataSourceRegister

Hi all, I've opened a WIP PR here https://github.com/apache/spark/pull/28159 I'm a novice at Scala, so I'm sure the code isn't idiomatic, but it behaves functionally how I'd expect. I've added unit tests to the PR, but if you would like to verify the intended functionality, I've uploaded a fat

Re: [VOTE] Apache Spark 3.0.0 RC1

The ongoing critical issues I'm aware of are: SPARK-31257 : Fix ambiguous two different CREATE TABLE syntaxes SPARK-31404 : backward compatibility issues after switching to Proleptic Gregorian

Re: [VOTE] Apache Spark 3.0.0 RC1

Only the low-risk or high-value bug fixes, and the documentation changes are allowed to merge to branch-3.0. I expect all the committers are following the same rules like what we did in the previous releases. Xiao On Thu, Apr 9, 2020 at 6:13 PM Jungtaek Lim wrote: > Looks like around 80

Re: [VOTE] Apache Spark 3.0.0 RC1

Looks like around 80 commits have been landed to branch-3.0 after we cut RC1 (I know many of them are to version the config, as well as add docs). Shall we announce the blocker-only phase and maintain the list of blockers to restrict the changes on the branch? This would make everyone being

Re: [VOTE] Apache Spark 3.0.0 RC1

I went through some manually tests for the new features of Structured Streaming in Spark 3.0.0. (Please let me know if there're more features we'd like to test manually.) * file source cleanup - both “archive" and “delete" work. Query fails as expected when the input directory is the output