You can try this .. But slightly modified the input structure since first two columns were not in Json format.
[image: Inline image 1] Thanks & Regards, Gokula Krishnan* (Gokul)* On Wed, Dec 23, 2015 at 9:46 AM, Eran Witkon <eranwit...@gmail.com> wrote: > Did you get a solution for this? > > On Tue, 22 Dec 2015 at 20:24 raja kbv <raja...@yahoo.com.invalid> wrote: > >> Hi, >> >> I am new to spark. >> >> I have a text file with below structure. >> >> >> (employeeID: Int, Name: String, ProjectDetails: JsonObject{[{ProjectName, >> Description, Duriation, Role}]}) >> Eg: >> (123456, Employee1, {“ProjectDetails”:[ >> { >> “ProjectName”: “Web Develoement”, “Description” : “Online Sales website”, >> “Duration” : “6 Months” , “Role” : “Developer”} >> { >> “ProjectName”: “Spark Develoement”, “Description” : “Online Sales >> Analysis”, “Duration” : “6 Months” , “Role” : “Data Engineer”} >> { >> “ProjectName”: “Scala Training”, “Description” : “Training”, “Duration” : >> “1 Month” } >> ] >> } >> >> >> Could someone help me to parse & flatten the record as below dataframe >> using scala? >> >> employeeID,Name, ProjectName, Description, Duration, Role >> 123456, Employee1, Web Develoement, Online Sales website, 6 Months , >> Developer >> 123456, Employee1, Spark Develoement, Online Sales Analysis, 6 Months, >> Data Engineer >> 123456, Employee1, Scala Training, Training, 1 Month, null >> >> >> Thank you in advance. >> >> Regards, >> Raja >> >