[ https://issues.apache.org/jira/browse/SPARK-40277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen resolved SPARK-40277. ---------------------------------- Resolution: Invalid This doesn't state any problem or specific change > Use DataFrame's column for referring to DDL schema for from_csv() and > from_json() > --------------------------------------------------------------------------------- > > Key: SPARK-40277 > URL: https://issues.apache.org/jira/browse/SPARK-40277 > Project: Spark > Issue Type: New Feature > Components: SQL > Affects Versions: 3.0.0 > Reporter: Jayant Kumar > Priority: Major > Original Estimate: 504h > Remaining Estimate: 504h > > With spark's DataFrame api one has to explicitly pass the StrucType to > functions like from_csv and from_json. This works okay in general. > In certain circumstances when schema depends on the one of the DataFrame's > field, it gets complicated and one has to switch to RDD. This requires > additional libraries to be added with additional parsing logic. > I am trying to explore a way to enable such use cases with DataFrame api and > function itself. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org