Re: Question about SPARK-11374 (skip.header.line.count)

Thank you for the opinion, Mingjie and Liang-Chi. Dongjoon. On Sun, Dec 11, 2016 at 5:42 PM, Liang-Chi Hsieh wrote: > Hi Dongjoon, > > I know some people only use Spark SQL with SQL syntax not Dataset API. So I > think it should be useful to provide a way to do this in SQL. >

Re: Question about SPARK-11374 (skip.header.line.count)

Hi Dongjoon, I know some people only use Spark SQL with SQL syntax not Dataset API. So I think it should be useful to provide a way to do this in SQL. - Liang-Chi Hsieh | @viirya Spark Technology Center -- View this message in context:

Re: Question about SPARK-11374 (skip.header.line.count)

hink it's useful to always have a pure SQL way and skip header for >> plain text / csv that lots of companies have. >> >> >> -- >> *From:* Dongjoon Hyun <dongj...@apache.org> >> *Sent:* Friday, December 9, 2016 9:42:58 AM >

Re: Question about SPARK-11374 (skip.header.line.count)

mpanies have. > > > -- > *From:* Dongjoon Hyun <dongj...@apache.org> > *Sent:* Friday, December 9, 2016 9:42:58 AM > *To:* Dongjin Lee; dev@spark.apache.org > *Subject:* Re: Question about SPARK-11374 (skip.header.line.count) > > Thank you for the opinion, Dongjin! > &g

Re: Question about SPARK-11374 (skip.header.line.count)

estion about SPARK-11374 (skip.header.line.count) Thank you for the opinion, Dongjin! On Thu, Dec 8, 2016 at 21:56 Dongjin Lee <dong...@apache.org<mailto:dong...@apache.org>> wrote: +1 For this idea. I need it also. Regards, Dongjin On Fri, Dec 9, 2016 at 8:59 AM, Dongjoon Hyun &l

Re: Question about SPARK-11374 (skip.header.line.count)

Thank you for the opinion, Dongjin! On Thu, Dec 8, 2016 at 21:56 Dongjin Lee wrote: > +1 For this idea. I need it also. > > Regards, > Dongjin > > On Fri, Dec 9, 2016 at 8:59 AM, Dongjoon Hyun wrote: > > Hi, All. > > > > > > Could you give me some

Fwd: Question about SPARK-11374 (skip.header.line.count)

+dev I forget to add @user. Dongjoon. -- Forwarded message - From: Dongjoon Hyun <dongj...@apache.org> Date: Thu, Dec 8, 2016 at 16:00 Subject: Question about SPARK-11374 (skip.header.line.count) To: <dev@spark.apache.org> Hi, All. Could you give me

Question about SPARK-11374 (skip.header.line.count)

Hi, All. Could you give me some opinion? There is an old SPARK issue, SPARK-11374, about removing header lines from text file. Currently, Spark supports removing CSV header lines by the following way. ``` scala> spark.read.option("header","true").csv("/data").show +---+---+ | c1| c2| +---+---+