[ https://issues.apache.org/jira/browse/SPARK-36950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-36950: --------------------------------- Issue Type: Improvement (was: Wish) > Normalize semi-structured data into a flat table. > ------------------------------------------------- > > Key: SPARK-36950 > URL: https://issues.apache.org/jira/browse/SPARK-36950 > Project: Spark > Issue Type: Improvement > Components: PySpark > Affects Versions: 3.3.0 > Reporter: Bjørn Jørgensen > Priority: Major > > Hi, in pandas there is this json_normalize that flat out nested data. > https://github.com/pandas-dev/pandas/blob/v1.3.3/pandas/io/json/_normalize.py#L112-L353 > > I have opened a request for this function at koalas. Now there are more > people that will have some function over to pyspark. > https://github.com/databricks/koalas/issues/2162 > This is also a function that geopandas are using. In the meantime I have > found a gist that has code that flattens out the whole dataframe. > https://gist.github.com/nmukerje/e65cde41be85470e4b8dfd9a2d6aed50 -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org