Performance issues while fetching data from Amazon S3

2018-10-29 Thread Walia, Jyotsana
Hi We have Apache Drill and Zookeeper running in Kubernetes cluster. We are using Drill to fetch data from S3 storage. We are using the S3 plugin for this. We are able to successfully fetch the data but it’s taking way too long. The data size is not more than 2GB. What can we do to improve the

Re: Performance issues while fetching data from Amazon S3

2018-10-29 Thread Pritesh Maker
Jyotsana There was a similar issue reported recently - https://issues.apache.org/jira/browse/DRILL-6814 - It could be related to your use case as well. We are investigating the cause now. Pritesh On Mon, Oct 29, 2018 at 11:56 AM Walia, Jyotsana < jyotsana.wa...@blackrock.com> wrote: > Hi > > We

Re: Performance issues while fetching data from Amazon S3

2018-10-30 Thread Arina Yelchiyeva
Though in both cases the problem is the same (slow performance) but Drilll setup is different. Jyotsana, you can share more details (including setup details, query profile etc) in the existing Jira or create new one and link both Jiras. Kind regards, Arina On Tue, Oct 30, 2018 at 6:21 AM Pritesh