Re: Renaming nested columns in dataframe

[Alexander Chermenin]

Hello.

 

I think you can use something like this:

 

 

df.select(

    struct(

        column("site.site_id").as("id"),

        column("site.site_name").as("name"),

        column("site.site_domain").as("domain"),

        column("site.site_cat").as("cat"),

        struct(

            column("site.publisher.site_pub_id").as("id"),

            column("site.publisher.site_pub_name").as("name")

        ).as("publisher")

    ).as("site")

)

 

Best regards, Alexander Chermenin.

Web: http://chermenin.ru

Mail: a...@chermenin.ru

 

 

 

16.05.2016, 14:58, "Prashant Bhardwaj" <prashant2006s...@gmail.com>:

Hi

 

How can I rename nested columns in dataframe through scala API? Like following schema

|-- site: struct (nullable = false)

|    |-- site_id: string (nullable = true)

|    |-- site_name: string (nullable = true)

|    |-- site_domain: string (nullable = true)

|    |-- site_cat: array (nullable = true)

|    |    |-- element: string (containsNull = false)

|    |-- publisher: struct (nullable = false)

|    |    |-- site_pub_id: string (nullable = true)

|    |    |-- site_pub_name: string (nullable = true)

 

I want to change it to  

|-- site: struct (nullable = false)

|    |-- id: string (nullable = true)

|    |-- name: string (nullable = true)

|    |-- domain: string (nullable = true)

|    |-- cat: array (nullable = true)

|    |    |-- element: string (containsNull = false)

|    |-- publisher: struct (nullable = false)

|    |    |-- id: string (nullable = true)

|    |    |-- name: string (nullable = true)

 

 

Regards

Prashant