Re: examples for flattening dataframes using pyspark

2017-05-27 Thread Zeming Yu
Sorry, sent the incomplete email by mistake. Here's the full email: > Hi, > > I need to flatten a nested dataframe and I' following this example: > https://docs.databricks.com/spark/latest/spark-sql/complex-types.html > > Just wondering: > 1. how can I test for the existence of an item before

examples for flattening dataframes using pyspark

2017-05-27 Thread Zeming Yu
Hi, I need to flatten a nested dataframe and I' following this example: https://docs.databricks.com/spark/latest/spark-sql/complex-types.html Just wondering: 1. how can I test for the existence of an item before retrieving it Say test if "b" exists before adding that into my flat dataframe