done :)
On Tue, Jul 21, 2015 at 6:16 PM, Jacques Nadeau <[email protected]> wrote:
> Whoa, something strange is going on.
>
> Expected
> select * from dfs.tmp.test2;
> +-----------------+
> | dimensions |
> +-----------------+
> | {"adults":"A"} |
> +-----------------+
>
> select p.dimensions as b from dfs.tmp.test2 p;
> +-----------------+
> | b |
> +-----------------+
> | {"adults":"A"} |
> +-----------------+
>
> select p.dimensions as b from dfs.tmp.test2 p;
> +-----------------+
> | b |
> +-----------------+
> | {"adults":"A"} |
> +-----------------+
>
> select p.dimensions.nonexist as a from dfs.tmp.test2 p;
> +-------+
> | b |
> +-------+
> | null |
> +-------+
>
> Unexpected
>
> select p.dimensions.nonexist as a, p.dimensions as b from dfs.tmp.test2 p;
> +----+-------+
> | a | b |
> +----+-------+
> | A | null |
> +----+-------+
>
> <<both wrong>>
>
> select p.dimensions as b, p.dimensions.nonexist as a from dfs.tmp.test2 p;
> +-------+-------+
> | b | a |
> +-------+-------+
> | null | null |
> +-------+-------+
>
> << b is wrong, a is correct>>
>
> Stefan, not sure what is going on here. It looks like an issue with
> resolution of a nonexistent field in a multilevel structure. It exists
> whether it is Parquet or JSON. Can you file a JIRA and mark it Critical?
>
>
> On Tue, Jul 21, 2015 at 10:46 AM, Stefán Baxter <[email protected]
> >
> wrote:
>
> > Hi,
> >
> > all kidding aside then this renders Drill+Parquet unusable for us at the
> > moment.
> >
> > If there is a quick fix then please let me know. :)
> >
> > Regards,
> > -Stefan
> >
> > On Tue, Jul 21, 2015 at 5:37 PM, Stefán Baxter <
> [email protected]>
> > wrote:
> >
> > > Well,
> > >
> > > After some more testing it appears that this has nothing to do with
> trim.
> > > (any non existing nested-value will be pushed aside)
> > >
> > > select p.dimensions.budgetLevel as `field1`, lower(p.dimensions.adults)
> > as
> > > `field2` from dfs.tmp.`/test/0_0_0.parquet` as p;
> > >
> > > also returns:
> > > +---------+---------+
> > > | field1 | field2 |
> > > +---------+---------+
> > > | a | null |
> > > +---------+---------+
> > >
> > > I just as puzzled though :)
> > >
> > > Regards,
> > > -Stefan
> > >
> > > On Tue, Jul 21, 2015 at 5:32 PM, Stefán Baxter <
> > [email protected]>
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> Here is small trick I think you will like :).
> > >>
> > >> - With this minimal dataset as /tmp/test.json:
> > >> {"dimensions":{"adults":"A"}}
> > >>
> > >> - Running this:
> > >> select lower(p.dimensions.budgetLevel) as `field1`,
> > >> lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test.json` as
> > p;
> > >>
> > >> - To no surprise returns this:
> > >> +---------+---------+
> > >> | field1 | field2 |
> > >> +---------+---------+
> > >> | null | a |
> > >> +---------+---------+
> > >>
> > >> Here comes the trick(y) part (hold your breath):
> > >>
> > >> - With the same data as a Parquet file
> > >> CREATE TABLE dfs.tmp.`/test` AS SELECT * FROM dfs.tmp.`/test.json`;
> > >>
> > >> - The same query:
> > >> select lower(p.dimensions.budgetLevel) as `field1`,
> > >> lower(p.dimensions.adults) as `field2` from
> > dfs.tmp.`/test/0_0_0.parquet`
> > >> as p;
> > >>
> > >> - Return this:
> > >> +---------+---------+
> > >> | field1 | field2 |
> > >> +---------+---------+
> > >> | a | null |
> > >> +---------+---------+
> > >>
> > >> ta ta !
> > >>
> > >> Best regards,
> > >> -Stefan
> > >>
> > >>
> > >>
> > >>
> > >
> >
>
