Re: 'dir0' not found in any table

2015-11-21 Thread Jacques Nadeau
I'm not seeing the problem. Can you provide more information (file type, exact commit, etc)? Below is what I get with 1.3rc3. $ tree /src/data/stuff /src/data/stuff ├── s1 │ └── 1.parquet └── s2 └── 1.parquet 0: jdbc:drill:zk=local> select dir0, count(*) from dfs.root.`/src/data/stuff`

Benefits of parquet partitioning for non-restrictive, aggregate queries?

2015-11-21 Thread Edmon Begoli
Hey guys, Are there any benefits of generic partitioning for non-restrictive count(*) queries with Drill and Parquet files partitioned on some base criteria (by state, month, etc.) Let's say I am running: select count(*) from dfs.tmp.`claims_parquet`; where I have plain and partitioned

Re: 'dir0' not found in any table

2015-11-21 Thread Stefán Baxter
thnx :) On Sat, Nov 21, 2015 at 11:00 PM, Jacques Nadeau wrote: > It looks DRILL-3810 caused the regression (and it is limited to Avro > files). Hopefully Kamesh can take a quick look. > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Sat, Nov 21, 2015 at 1:44 PM,

Re: 'dir0' not found in any table

2015-11-21 Thread Stefán Baxter
Hi, I just created this: https://issues.apache.org/jira/browse/DRILL-4120 Regards, -Stefan On Sat, Nov 21, 2015 at 9:28 PM, Stefán Baxter wrote: > > After some digging around there is an explanation. > > This all works fine when the directory structure contains

CTAS - Converting Avro files to parquet - Missing timestamp datatype

2015-11-21 Thread Stefán Baxter
Hi, We are using Avro files for all our logging and they contain long timestamp_mills values. When they are converted to Parquet using CTAS we wither need a hint (or something) to ensure that these columns become Timestamp values in parquet - or - we need to create a complex select with casting.

Re: 'dir0' not found in any table

2015-11-21 Thread Jacques Nadeau
It looks DRILL-3810 caused the regression (and it is limited to Avro files). Hopefully Kamesh can take a quick look. -- Jacques Nadeau CTO and Co-Founder, Dremio On Sat, Nov 21, 2015 at 1:44 PM, Stefán Baxter wrote: > Hi, > > I just created this:

Re: 'dir0' not found in any table

2015-11-21 Thread Stefán Baxter
After some digging around there is an explanation. This all works fine when the directory structure contains Parquet files but it breaks is the same structure contains Avro files. I have no clue to why that is and I hope you do. -Stefan On Sat, Nov 21, 2015 at 3:58 PM, Jacques Nadeau

Re: Querying k/v pairs in a CSV column

2015-11-21 Thread Christopher Matta
Along the same lines I have a data set that's delimited by pipe with the last column further delimited by commas. It would be great to be able to flatten that last column. On Wednesday, October 7, 2015, Tugdual Grall wrote: > Hi, > > Can you clarify how to use our own format