Maxim Gekk created SPARK-31020: ---------------------------------- Summary: Support foldable schemas by `from_csv` Key: SPARK-31020 URL: https://issues.apache.org/jira/browse/SPARK-31020 Project: Spark Issue Type: New Feature Components: SQL Affects Versions: 3.0.0 Reporter: Maxim Gekk
Currently, Spark accepts only literals or schema_of_csv w/ literal input as the schema parameter of from_csv. And it fails on any foldable expressions, for instance: {code:sql} spark-sql> select from_csv('1, 3.14', replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', '')); Error in query: Schema should be specified in DDL format as a string literal or output of the schema_of_csv function instead of replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', '');; line 1 pos 7 {code} There is reasons to restrict users by literals. The ticket aims to support any foldable schemas by from_csv(). -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org