rxin commented on pull request #31569:
URL: https://github.com/apache/spark/pull/31569#issuecomment-822969944
With all due respect, the number of functions is not a good measure. Those
functions are the most fundamental ones in the DataFrame API. Basically any
time anybody wants to refer to a column... just look up stack over low. There
are tons of answers with sample code doing ‘col.
I don’t think Scala should remove the symbol literal syntax given its
damage. It looks like it’s possible it will be kept with an import and then
Spark users wouldn’t need to suffer.
On Mon, Apr 19 2021 at 9:42 PM, Kousuke Saruta < ***@***.*** > wrote:
>
>
>
>
>
>
>>
>>
>> Public Spark APIs that use Symbols. We can't change them as they are
>> public APIs. We are fine if Scala won't remove the Symbol class. I assume
>> these public APIs will be less popular over time, as they are hard to use
>> without the symbol literal syntax. $"col_name" is better.
>>
>>
>
>
>
> Fortunately, the usage of Symbols in public non-internal APIs is limited.
>
>
>
> * as and alias in Dataset.scala
> * as in Column.scala
> * typedlit in functions.scala
>
>
>
>
> So I think we can decide to deprecate these APIs later.
>
>
>>
>>
>> Internal Spark APIs that use Symbols, especially the testing DSL. I'd
>> prefer to not use Symbol anymore, as it becomes hard to use without the
>> symbol literal syntax. e.g. LocalRelation('a.int) looks good, but
>> LocalRelation(Symbol("a").int) looks cumbersome. We can create a new DSL
>> syntax to make it easier to write tests in Spark. (see #31601 (
>> https://github.com/apache/spark/pull/31601 ) )
>>
>>
>
>
>
> As I noticed in #31601 ( https://github.com/apache/spark/pull/31601 ) ,
> most usage of Symbols seems to be in testing DSL.
> Symbol constructor is used a little but most is symbol literals.
> So I think we can safely replace them with another syntax.
>
>
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub (
> https://github.com/apache/spark/pull/31569#issuecomment-822968797 ) , or
unsubscribe
> (
>
https://github.com/notifications/unsubscribe-auth/AACO6PHFWQSACDOYRATSPXLTJUA5BANCNFSM4XVVLARQ
> ).
>
>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
