Hi, I met a problem of empty field in the nested JSON file with Spark SQL. For instance, There are two lines of JSON file as follows,
{ "firstname": "Jack", "lastname": "Nelson", "address": { "state": "New York", "city": "New York" } }{ "firstname": "Landy", "middlename": "Ken", "lastname": "Yong", "address": { "state": "California", "city": "Los Angles" } } I use Spark SQL to get the files like, val row = sqlContext.sql("SELECT firstname, middlename, lastname, address.state, address.city FROM jsontable") The compile will tell me the error of line1: no "middlename". How do I handle this case in the SQL sql? Many thanks in advance! Jerry