Re: [HANGOUT] Topics for 7/25/17

2017-07-25 Thread yuliya Feldman
Sorry for the late chime in.Just a note - regarding s3 - even after upgrade to hadoop 2.8.x you may need to separately update versions of aws, as one provided with the upgrade is not supporting all the newly added regions. Thanks,Yuliya From: Arina Yelchiyeva To: d...@drill.apache.org C

Re: append data to already existing table saved in parquet format

2017-07-25 Thread rahul challapalli
I am not aware of any clean way to do this. However if your data is partitioned based on directories, then you can use the below hack which leverages temporary tables [1]. Essentially, you backup your partition to a temp table, then override it by taking the union of new partition data and existing

Re: append data to already existing table saved in parquet format

2017-07-25 Thread Abhishek Girish
Drill doesn't have support for an insert into command. You could try using the CTAS command to write to a specific partition directory, may be? Also look at CTAS auto partitioning [1] [1] https://drill.apache.org/docs/partition-by-clause/ On Tue, Jul 25, 2017 at 10:52 PM, Divya Gehlot wrote: >

append data to already existing table saved in parquet format

2017-07-25 Thread Divya Gehlot
Hi, I am naive to Apache drill. As I have data coming in every hour , when I searched I couldnt find the insert into partition command in Apache drill. How can we insert data to particular partition without rewriting the whole data set ? Appreciate the help. Thanks, Divya

Re: Question about Drill aggregate queries and schema change

2017-07-25 Thread Cliff Resnick
Awesome! I'll be watching that issue for the PR. On Tue, Jul 25, 2017 at 2:50 PM, Jinfeng Ni wrote: > I'm currently working on a patch using the idea described in DRILL-5546. > The idea is similar to your idea of null row : in stead of returning an > empty batch, or a scan batch with injected nu

Re: Drill and Tableau

2017-07-25 Thread Saurabh Mahapatra
Hi Divya, There is nothing as a naive question. Please feel free to post any questions you have. There is someone in the community that will help you out. This is my opinion: There are a variety of BI tools in the market that offer excellent visualization and interaction with data capabilities. T

Re: Drill and Tableau

2017-07-25 Thread Jinfeng Ni
Some of questions you have are related to which type of storage you are querying with Drill. If you are querying hive table, Drill uses Hive metastore API to get the metadata, and internally maintain a cache for speedup [1]. If you are querying parquet files through file system plugin, Drill has th

Re: Question about Drill aggregate queries and schema change

2017-07-25 Thread Jinfeng Ni
I'm currently working on a patch using the idea described in DRILL-5546. The idea is similar to your idea of null row : in stead of returning an empty batch, or a scan batch with injected nullable-int columns, we will return NONE to the downstream operators directly, which will avoid the unintended

Re: [HANGOUT] Topics for 7/25/17

2017-07-25 Thread Arina Yelchiyeva
Meeting minutes 25 July 2017: Attendees: Rob, Vova, Sorabh, Pritesh, Paul, Aman, Padma, Jyothsna, Sindhuri. Two topics were discussed. 1. Release candidate for 1.11.0. Everybody is encouraged to test the release candidate and vote. Aman asked about the release candidate performance testing. Asked

Re: [HANGOUT] Topics for 7/25/17

2017-07-25 Thread Arina Yelchiyeva
Hangouts is starting now... On Tue, Jul 25, 2017 at 7:41 AM, Padma Penumarthy wrote: > I have a topic to discuss. Lot of folks on the user mailing list raised > the issue of not being able to access all S3 regions using Drill. > We need hadoop version 2.8 or higher to be able to connect to > reg

Drill and Tableau

2017-07-25 Thread Divya Gehlot
Hi, As a naive user would like to know the benefitsof Apache Drill with tableau ? As per my understanding we to visualize we need to push the data to tableau for granular visualization . Would like to understand few features of Drill in terms of visualtion or data retrieval : 1, Metadata Caching