Seems like you are also looking for transform and reduce for arrays?
https://issues.apache.org/jira/browse/SPARK-23908
https://issues.apache.org/jira/browse/SPARK-23911
On Wed, Jun 20, 2018 at 10:43 AM bobotu wrote:
> I store some trajectories data in parquet with this schema:
>
> create
I store some trajectories data in parquet with this schema:
create table traj (
id string,
points array>
>>
)
It contains a lots of attribute comes from sensors. It also have a nested
array which contains information generated during map-matching algorithm.
All of my
