sarutak commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-822968797
> Public Spark APIs that use Symbols. We can't change them as they are public APIs. We are fine if Scala won't remove the Symbol class. I assume these public APIs will be less popular over time, as they are hard to use without the symbol literal syntax. $"col_name" is better. Fortunately, the usage of Symbols in public non-internal APIs is limited. * as and alias in Dataset.scala * as in Column.scala * typedlit in functions.scala So I think we can decide to deprecate these APIs later. > Internal Spark APIs that use Symbols, especially the testing DSL. I'd prefer to not use Symbol anymore, as it becomes hard to use without the symbol literal syntax. e.g. LocalRelation('a.int) looks good, but LocalRelation(Symbol("a").int) looks cumbersome. We can create a new DSL syntax to make it easier to write tests in Spark. (see #31601) As I noticed in #31601, most usage of Symbols seems to be in testing DSL. `Symbol` constructor is used a little but most is symbol literals. So I think we can safely replace them with another syntax. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org