Hello,
If I have posted this question in the wrong mailing list, kindly suggest
the right one because I have just joined the group.
My JSON is in the following format:
{"ProfileId":11111111111,"CustomerId":111,"UserId":222,"DateString":"2011-01-01
14:24:52","Machine":"MyMachine","AppName":"TTT","Children":[{"Name":"Sk1","DurWithoutChildrenMs":21.23,"ParentId":11111111111,"IsSql":"True"},{"Name":"Sk2","DurWithoutChildrenMs":22.23,"ParentId":11111111111,"IsSql":"True"}]}
{"ProfileId":22222222222,"CustomerId":222,"UserId":333,"DateString":"2011-01-01
11:24:52","Machine":"MyMachine","AppName":"TTT","Children":[{"Name":"Sk3","DurWithoutChildrenMs":11.23,"ParentId":22222222222,"IsSql":"True"},{"Name":"Sk2","DurWithoutChildrenMs":32.23,"ParentId":22222222222,"IsSql":"True"}]}
I created a table with Array<Struct> as one of the columns for the children
array.
Create table BrokerLogsv2
(CustomerId int,
UserId int,
DateString string,
AppName string,
Machine string,
ProfileId bigint,
Children array<struct<Name:string, DurWithoutChildrenMs:float, IsSql:
boolean, ParentId:bigint>>
) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
When I load the data, all the fields are populated except for the Children
array.
I am not sure if array<struct> is possible with json. If not can somebody
please suggest some other way of loading the children array such that I can
explode the children node and perform some aggregate functions on it.
My plan is to do explode(children) using a lateral view?
Thanks,
Nac
