Need help in Apache Drill

2018-07-02 Thread dony.natrajan
Hi there, Hope you're doing good. I need suggestion on performance issues which I'm facing in Apache Drill. Apache Drill version - 1.12.0 DB - MongoDB Details: 1. For us, it took 9.530 seconds for 1 M records within Apache Drill. 2. Hence, the response time to load pa

Re: Web console responsiveness under heavy load

2018-07-02 Thread Kunal Khatua
Yes. The reason you see the sluggishness is because the embedded webserver within the Drillbit also acts as a proxy client, which receives the entire result set, before converting it into a JSON response for your browser. This can lead to rather high memory requirements.  I'd recommend using to

Re: CTAS AccessControlException

2018-07-02 Thread Abhishek Girish
Hey Divya, Here is one way to check if all nodes have the same UID/GID: clush -a 'cat /etc/passwd | grep -i user1' Node1: user1:x:5000:5000::/home/user1:/bin/bash Node2: user1:x:5000:5000::/home/user1:/bin/bash Node3: user1:x:6000:6000::/home/user1:/bin/bash You can update the UID and GID using

Re: CTAS AccessControlException

2018-07-02 Thread Divya Gehlot
Hi Abhishek, Thanks for the prompt response ! Yes I have Big Data Cluster and Apache Drill is part of it and security is plain authentication and connected through AD . And Recently I have added 3 more nodes to the cluster. How do I ensure that all the nodes have same UID + GID , which you mentione

unable to read timestamp when create CSV file wheras parquet can read

2018-07-02 Thread Divya Gehlot
Hi, I am getting error when I creating the CSV files The part of the query throwing error is as below, the input data comes in unix time format CAST(FROM_UNIXTIME(t.`timestamp`/1000) AS TIMESTAMP) AS `datetime` Apache Drill org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: Ille

Re: Web console responsiveness under heavy load

2018-07-02 Thread Dave Challis
Yup, running the query through the web UI. If I avoid doing that, would the UI be more responsive? Drillbit settings are: DRILL_HEAP=8G DRILL_MAX_DIRECT_MEMORY=16G DRILLBIT_CODE_CACHE_SIZE=2G The query itself uses a peak of ~3.5Gb memory while it's running. On 30 June 2018 at 08:18, Kunal Khatu

Re: Need help in Apache Drill

2018-07-02 Thread Kunal Khatua
Hi  Could you upload the screenshots to a public hosting site and post the links? The Apache mailing list blocks attachments for security reasons. ~ Kunal On 7/1/2018 10:22:00 PM, dony.natra...@ramyamlab.com wrote: Hi there,   Hope you’re doing good.   I need suggestion on performance issues w