Re: [DISCUSS] [Spark SQL, PySpark] Combining StructTypes into a new StructType

Thanks for the idea! I am thinking that the usage of "combined = StructType( a.fields + b.fields)" is still good because 1) it is not horrible to merge a and b in this way. 2) itself clarifies the intention which is merge two struct's fields to construct a new struct 3) you also have room to

Re: Welcome Xinrong Meng as a Spark committer

Congrats Xinrong! -Rui On Tue, Aug 9, 2022 at 8:57 PM Xingbo Jiang wrote: > Congratulations！ > > Yuanjian Li 于2022年8月9日 周二20:31写道： > >> Congratulations, Xinrong! >> >> XiDuo You 于2022年8月9日 周二19:18写道： >> >>> Congratulations! >>> >>> Haejoon Lee 于2022年8月10日周三 09:30写道： >>> > >>> > Congrats,

Re: Welcome Xinrong Meng as a Spark committer

Congratulations！ Yuanjian Li 于2022年8月9日 周二20:31写道： > Congratulations, Xinrong! > > XiDuo You 于2022年8月9日 周二19:18写道： > >> Congratulations! >> >> Haejoon Lee 于2022年8月10日周三 09:30写道： >> > >> > Congrats, Xinrong!! >> > >> > On Tue, Aug 9, 2022 at 5:12 PM Hyukjin Kwon >> wrote: >> >> >> >> Hi all, >>

Re: Welcoming three new PMC members

Congrats everyone! L. C. Hsieh 于2022年8月9日 周二19:01写道： > Congrats! > > On Tue, Aug 9, 2022 at 5:38 PM Chao Sun wrote: > > > > Congrats everyone! > > > > On Tue, Aug 9, 2022 at 5:36 PM Dongjoon Hyun > wrote: > > > > > > Congrat to all! > > > > > > Dongjoon. > > > > > > On Tue, Aug 9, 2022 at 5:13

Re: Welcome Xinrong Meng as a Spark committer

Congratulations, Xinrong! XiDuo You 于2022年8月9日 周二19:18写道： > Congratulations! > > Haejoon Lee 于2022年8月10日周三 09:30写道： > > > > Congrats, Xinrong!! > > > > On Tue, Aug 9, 2022 at 5:12 PM Hyukjin Kwon wrote: > >> > >> Hi all, > >> > >> The Spark PMC recently added Xinrong Meng as a committer on the

Re: Welcome Xinrong Meng as a Spark committer

Congratulations! Haejoon Lee 于2022年8月10日周三 09:30写道： > > Congrats, Xinrong!! > > On Tue, Aug 9, 2022 at 5:12 PM Hyukjin Kwon wrote: >> >> Hi all, >> >> The Spark PMC recently added Xinrong Meng as a committer on the project. >> Xinrong is the major contributor of PySpark especially Pandas API

Re: Welcoming three new PMC members

Congrats! On Tue, Aug 9, 2022 at 5:38 PM Chao Sun wrote: > > Congrats everyone! > > On Tue, Aug 9, 2022 at 5:36 PM Dongjoon Hyun wrote: > > > > Congrat to all! > > > > Dongjoon. > > > > On Tue, Aug 9, 2022 at 5:13 PM Takuya UESHIN wrote: > > > > > > Congratulations! > > > > > > On Tue, Aug 9,

Re: Welcome Xinrong Meng as a Spark committer

Congrats, Xinrong!! On Tue, Aug 9, 2022 at 5:12 PM Hyukjin Kwon wrote: > Hi all, > > The Spark PMC recently added Xinrong Meng as a committer on the project. > Xinrong is the major contributor of PySpark especially Pandas API on Spark. > She has guided a lot of new contributors

Re: Welcome Xinrong Meng as a Spark committer

Congratulations! On Tue, Aug 9, 2022 at 1:00 PM huaxin gao wrote: > > Congratulations! > > On Tue, Aug 9, 2022 at 12:47 PM Dongjoon Hyun wrote: >> >> Congrat! :) >> >> Dongjoon. >> >> On Tue, Aug 9, 2022 at 10:40 AM Takuya UESHIN wrote: >> > >> > Congratulations, Xinrong! >> > >> > On Tue, Aug

Re: Welcoming three new PMC members

Congrats everyone! On Tue, Aug 9, 2022 at 5:36 PM Dongjoon Hyun wrote: > > Congrat to all! > > Dongjoon. > > On Tue, Aug 9, 2022 at 5:13 PM Takuya UESHIN wrote: > > > > Congratulations! > > > > On Tue, Aug 9, 2022 at 4:57 PM Hyukjin Kwon wrote: > >> > >> Congrats everybody! > >> > >> On Wed,

Re: Welcoming three new PMC members

Congrat to all! Dongjoon. On Tue, Aug 9, 2022 at 5:13 PM Takuya UESHIN wrote: > > Congratulations! > > On Tue, Aug 9, 2022 at 4:57 PM Hyukjin Kwon wrote: >> >> Congrats everybody! >> >> On Wed, 10 Aug 2022 at 05:50, Mridul Muralidharan wrote: >>> >>> >>> Congratulations ! >>> Great to have

Re: Welcoming three new PMC members

Congratulations! On Tue, Aug 9, 2022 at 4:57 PM Hyukjin Kwon wrote: > Congrats everybody! > > On Wed, 10 Aug 2022 at 05:50, Mridul Muralidharan > wrote: > >> >> Congratulations ! >> Great to have you join the PMC !! >> >> Regards, >> Mridul >> >> On Tue, Aug 9, 2022 at 11:57 AM vaquar khan >>

Re: Welcoming three new PMC members

Congrats everyone! On Wed, Aug 10, 2022 at 8:57 AM Hyukjin Kwon wrote: > Congrats everybody! > > On Wed, 10 Aug 2022 at 05:50, Mridul Muralidharan > wrote: > >> >> Congratulations ! >> Great to have you join the PMC !! >> >> Regards, >> Mridul >> >> On Tue, Aug 9, 2022 at 11:57 AM vaquar khan

Re: Welcoming three new PMC members

Congrats everybody! On Wed, 10 Aug 2022 at 05:50, Mridul Muralidharan wrote: > > Congratulations ! > Great to have you join the PMC !! > > Regards, > Mridul > > On Tue, Aug 9, 2022 at 11:57 AM vaquar khan wrote: > >> Congratulations >> >> On Tue, Aug 9, 2022, 11:40 AM Xiao Li wrote: >> >>> Hi

Re: Welcoming three new PMC members

Congratulations ! Great to have you join the PMC !! Regards, Mridul On Tue, Aug 9, 2022 at 11:57 AM vaquar khan wrote: > Congratulations > > On Tue, Aug 9, 2022, 11:40 AM Xiao Li wrote: > >> Hi all, >> >> The Spark PMC recently voted to add three new PMC members. Join me in >> welcoming them

[DISCUSS] [Spark SQL, PySpark] Combining StructTypes into a new StructType

Hi all, this is my first message to the Spark mailing list, so please bear with me if I don't fully meet your communication standards. I just wanted to discuss one aspect that I've stumbled across several times over the past few weeks. When working with Spark, I often run into the problem of

Re: Welcome Xinrong Meng as a Spark committer

Congratulations! On Tue, Aug 9, 2022 at 12:47 PM Dongjoon Hyun wrote: > Congrat! :) > > Dongjoon. > > On Tue, Aug 9, 2022 at 10:40 AM Takuya UESHIN > wrote: > > > > Congratulations, Xinrong! > > > > On Tue, Aug 9, 2022 at 10:07 AM Gengliang Wang wrote: > >> > >> Congratulations, Xinrong! Well

Re: Welcome Xinrong Meng as a Spark committer

Congrat! :) Dongjoon. On Tue, Aug 9, 2022 at 10:40 AM Takuya UESHIN wrote: > > Congratulations, Xinrong! > > On Tue, Aug 9, 2022 at 10:07 AM Gengliang Wang wrote: >> >> Congratulations, Xinrong! Well deserved. >> >> >> On Tue, Aug 9, 2022 at 7:09 AM Yi Wu wrote: >>> >>> Congrats Xinrong!! >>>

Re: Welcome Xinrong Meng as a Spark committer

Congratulations, Xinrong! On Tue, Aug 9, 2022 at 10:07 AM Gengliang Wang wrote: > Congratulations, Xinrong! Well deserved. > > > On Tue, Aug 9, 2022 at 7:09 AM Yi Wu wrote: > >> Congrats Xinrong!! >> >> >> On Tue, Aug 9, 2022 at 7:07 PM Maxim Gekk >> wrote: >> >>> Congratulations, Xinrong!

Allowing all Reader or Writer settings to be provided as options

Hello people, I want to bring some attention to SPARK-39630 and ask if there are any design objections to the idea proposed there. The gist of the proposal is that there are some reader or writer directives that cannot be supplied as

Re: Welcome Xinrong Meng as a Spark committer

Congratulations, Xinrong! Well deserved. On Tue, Aug 9, 2022 at 7:09 AM Yi Wu wrote: > Congrats Xinrong!! > > > On Tue, Aug 9, 2022 at 7:07 PM Maxim Gekk > wrote: > >> Congratulations, Xinrong! >> >> Maxim Gekk >> >> Software Engineer >> >> Databricks, Inc. >> >> >> On Tue, Aug 9, 2022 at

Re: Welcoming three new PMC members

Congratulations On Tue, Aug 9, 2022, 11:40 AM Xiao Li wrote: > Hi all, > > The Spark PMC recently voted to add three new PMC members. Join me in > welcoming them to their new roles! > > New PMC members: Huaxin Gao, Gengliang Wang and Maxim Gekk > > The Spark PMC >

Welcoming three new PMC members

Hi all, The Spark PMC recently voted to add three new PMC members. Join me in welcoming them to their new roles! New PMC members: Huaxin Gao, Gengliang Wang and Maxim Gekk The Spark PMC

Re: Welcome Xinrong Meng as a Spark committer

Congrats Xinrong!! On Tue, Aug 9, 2022 at 7:07 PM Maxim Gekk wrote: > Congratulations, Xinrong! > > Maxim Gekk > > Software Engineer > > Databricks, Inc. > > > On Tue, Aug 9, 2022 at 3:15 PM Weichen Xu > wrote: > >> Congrats! >> >> On Tue, Aug 9, 2022 at 5:55 PM Jungtaek Lim >> wrote: >> >>>

Re: Welcome Xinrong Meng as a Spark committer

Congratulations, Xinrong! Maxim Gekk Software Engineer Databricks, Inc. On Tue, Aug 9, 2022 at 3:15 PM Weichen Xu wrote: > Congrats! > > On Tue, Aug 9, 2022 at 5:55 PM Jungtaek Lim > wrote: > >> Congrats Xinrong! Well deserved. >> >> 2022년 8월 9일 (화) 오후 5:13, Hyukjin Kwon 님이 작성: >> >>> Hi

Re: Welcome Xinrong Meng as a Spark committer

Congrats! On Tue, Aug 9, 2022 at 5:55 PM Jungtaek Lim wrote: > Congrats Xinrong! Well deserved. > > 2022년 8월 9일 (화) 오후 5:13, Hyukjin Kwon 님이 작성: > >> Hi all, >> >> The Spark PMC recently added Xinrong Meng as a committer on the project. >> Xinrong is the major contributor of PySpark especially

Re: Welcome Xinrong Meng as a Spark committer

Congrats Xinrong! Well deserved. 2022년 8월 9일 (화) 오후 5:13, Hyukjin Kwon 님이 작성: > Hi all, > > The Spark PMC recently added Xinrong Meng as a committer on the project. > Xinrong is the major contributor of PySpark especially Pandas API on Spark. > She has guided a lot of new contributors

Re: Welcome Xinrong Meng as a Spark committer

Congratulations Xinrong! Regards, Qian SUN Yang,Jie(INF) 于2022年8月9日周二 17:10写道： > Congratulations! > > > Regards, > > Yang Jie > > > > > > *发件人**: *Hyukjin Kwon > *日期**: *2022年8月9日 星期二 16:12 > *收件人**: *dev > *主题**: *Welcome Xinrong Meng as a Spark committer > > > > Hi all, > > > > The Spark

Re: Welcome Xinrong Meng as a Spark committer

Congratulations! Regards, Yang Jie 发件人: Hyukjin Kwon 日期: 2022年8月9日 星期二 16:12 收件人: dev 主题: Welcome Xinrong Meng as a Spark committer Hi all, The Spark PMC recently added Xinrong Meng as a committer on the project. Xinrong is the major contributor of PySpark especially Pandas API on Spark.

Re: Welcome Xinrong Meng as a Spark committer

Congratulations! Regards, Yikun On Tue, Aug 9, 2022 at 4:13 PM Hyukjin Kwon wrote: > Hi all, > > The Spark PMC recently added Xinrong Meng as a committer on the project. > Xinrong is the major contributor of PySpark especially Pandas API on Spark. > She has guided a lot of new contributors

Re: Welcome Xinrong Meng as a Spark committer

Congratulations Xinrong ! Regards, Mridul On Tue, Aug 9, 2022 at 3:13 AM Hyukjin Kwon wrote: > Hi all, > > The Spark PMC recently added Xinrong Meng as a committer on the project. > Xinrong is the major contributor of PySpark especially Pandas API on Spark. > She has guided a lot of new

Welcome Xinrong Meng as a Spark committer

Hi all, The Spark PMC recently added Xinrong Meng as a committer on the project. Xinrong is the major contributor of PySpark especially Pandas API on Spark. She has guided a lot of new contributors enthusiastically. Please join me in welcoming Xinrong!