Here's a WIP PR with the basic changes: https://github.com/apache/spark/pull/28026
I still need to update tests in that branch and add the conversions to the old Hive plans. But at least you can see how the parser part works and how I'm converting the extra clauses for DSv2. This also enables us to support Hive create syntax in DSv2. On Wed, Mar 25, 2020 at 3:59 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote: > Would it be better to prioritize this to make sure the change is included > in Spark 3.0? (Maybe filing an issue and set as a blocker) > > Looks like there's consensus that SPARK-30098 brought ambiguous issue > which should be fixed (though the consideration of severity seems to be > different), and once we notice the issue it would be really odd if we > publish it as it is, and try to fix it later (the fix may not be even > included in 3.0.x as it might bring behavioral change). > > On Tue, Mar 24, 2020 at 3:37 PM Wenchen Fan <cloud0...@gmail.com> wrote: > >> Hi Ryan, >> >> It's great to hear that you are cleaning up this long-standing mess. >> Please let me know if you hit any problems that I can help with. >> >> Thanks, >> Wenchen >> >> On Sat, Mar 21, 2020 at 3:16 AM Nicholas Chammas < >> nicholas.cham...@gmail.com> wrote: >> >>> On Thu, Mar 19, 2020 at 3:46 AM Wenchen Fan <cloud0...@gmail.com> wrote: >>> >>>> 2. PARTITIONED BY colTypeList: I think we can support it in the >>>> unified syntax. Just make sure it doesn't appear together with PARTITIONED >>>> BY transformList. >>>> >>> >>> Another side note: Perhaps as part of (or after) unifying the CREATE >>> TABLE syntax, we can also update Catalog.createTable() to support >>> creating partitioned tables >>> <https://issues.apache.org/jira/browse/SPARK-31001>. >>> >> -- Ryan Blue Software Engineer Netflix
