Re: [DISCUSS][SQL][PySpark] Column name support for SQL functions

2019-03-06 Thread Reynold Xin
> >   > > > > *From:* Reynold Xin < r...@databricks.com > > *Date:* Monday, 25 February 2019 at 00:49 > *To:* Felix Cheung < felixcheun...@hotmail.com > > *Cc:* dev < dev@spark.apache.org >, Sean Owen < sro...@gmail.com >, André > Mello

Re: [DISCUSS][SQL][PySpark] Column name support for SQL functions

2019-03-04 Thread André Mello
___ From: Sean Owen mailto:sro...@gmail.com>> Sent: Sunday, February 24, 2019 6:59 AM To: André Mello Cc: dev Subject: Re: [DISCUSS][SQL][PySpark] Column name support for SQL functions I just commented on the PR -- I personally don't think it's worth removing support for, say, max("

Re: [DISCUSS][SQL][PySpark] Column name support for SQL functions

2019-02-24 Thread Reynold Xin
gt; (My 2 c) > > > >   > *From:* Sean Owen < srowen@ gmail. com ( sro...@gmail.com ) > > *Sent:* Sunday, February 24, 2019 6:59 AM > *To:* André Mello > *Cc:* dev > *Subject:* Re: [DISCUSS][SQL][PySpark] Column name support for SQL > functions >   > I just

Re: [DISCUSS][SQL][PySpark] Column name support for SQL functions

2019-02-24 Thread Felix Cheung
) From: Sean Owen Sent: Sunday, February 24, 2019 6:59 AM To: André Mello Cc: dev Subject: Re: [DISCUSS][SQL][PySpark] Column name support for SQL functions I just commented on the PR -- I personally don't think it's worth removing support for, say, max("foo") over max(col(&quo

Re: [DISCUSS][SQL][PySpark] Column name support for SQL functions

2019-02-24 Thread Sean Owen
I just commented on the PR -- I personally don't think it's worth removing support for, say, max("foo") over max(col("foo")) or max($"foo") in Scala. We can make breaking changes in Spark 3 but this seems like it would unnecessarily break a lot of code. The string arg is more concise in Python and

[DISCUSS][SQL][PySpark] Column name support for SQL functions

2019-02-24 Thread André Mello
# Context This comes from [SPARK-26979], which became PR #23879 and then PR #23882. The following reflects all the findings made so far. # Description Currently, in the Scala API, some SQL functions have two overloads, one taking a string that names the column to be operated on, the other