What does the jstack command (located in the same dir as the java executable) say when a query is hung?
Also, what is the JStack when the Drillbit is running with the exec.queue.enable=FALSE ? ________________________________ From: jasbir.s...@accenture.com <jasbir.s...@accenture.com> Sent: Thursday, May 4, 2017 5:52:43 AM To: user@drill.apache.org Cc: maneesh.koth...@accenture.com; h.p.ku...@accenture.com Subject: RE: Drill query are stuck in ENQUEUED mode Hi, Thanks for the updates. Just want to update you that we are already following this parameter. Value of below parameters are - 1. exec.queue.enable = TRUE 2. exec.queue.large = 10 (Default) 3. exec.queue.small = 100 (Default) 4. exec.queue.threshold = 30000000(Default) Exec.queue.timeout_millis = 300000(Default) Even after following queries are getting stuck. As in our case parquet record count is less than 1000, do I still need to set this threshold or I need to change this value? As it doesn't tells me on how much record count it decides to queue the query in SMALL section. What is that record count? Description of my cluster is - 1. Nodes = 1 - Local mode 2. Memory = 64 GB 3. Cores = 24 4. It's a linux box Please do let me know how to resolve query stuck issue as it is hampering products performance. Regards, Jasbir Singh -----Original Message----- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: Wednesday, May 03, 2017 10:43 PM To: user@drill.apache.org Cc: Kothari, Maneesh <maneesh.koth...@accenture.com>; Kumar, H. P. <h.p.ku...@accenture.com> Subject: Re: Drill query are stuck in ENQUEUED mode You could try experimenting with these parameters: https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_enabling-2Dquery-2Dqueuing_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=Dr7zjb2Guf7ZBOsBVweE4jbrxUv_RRxAMGk9NBqL5Zo&e= when you have Queuing enabled. However, in a false state, there should be no queuing (i.e. no ENQUEUED mode). Can you provide a description of your cluster? (e.g. nodes, cores, memory... is it a VM, etc) One way to understand why a query is 'stuck' with queuing disabled would be to use JStack and identify the state of the foreman and fragment threads executing that query. For this, you first need to know what is the query ID (the part that follows the "https://urldefense.proofpoint.com/v2/url?u=http-3A__hostname-3A8047_profiles_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=WL1ut57ugpifNnM-IVN1E-NzFBOigrMvDwHw4Uj3etg&e= " part of the query's web UI page). Use that as a reference to identify the corresponding threads. Usually, a large number of threads will carry this as a part of their thread names. However, if it is stuck, my hunch is that only 1 fragment thread is stuck, while the others completed. So you should be able to find that easily. Kunal Khatua Engineering [1490734684477_mapr.png] www.mapr.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mapr.com_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=sYU_DLIUmwFTX0OxodY-x06c7dD7RrXjNex3CQFiQVk&e= > ________________________________ From: jasbir.s...@accenture.com <jasbir.s...@accenture.com> Sent: Wednesday, May 3, 2017 9:37:44 AM To: user@drill.apache.org Cc: maneesh.koth...@accenture.com; h.p.ku...@accenture.com Subject: RE: Drill query are stuck in ENQUEUED mode Few things to update - My parquet files are having less than 1000 records. I create around 250 parquet files from my application and fetch data from them using Apache Drill. When I restart the drill instance all 250 queries on parquet files would execute, but after sometime out of 250 only 1 query would be stuck. And this process repeats thereafter after every execution. And can someone also let me know about the number of records which determine that it will be in exec.queue.small? Regards, Jasbir Singh -----Original Message----- From: Sing, Jasbir Sent: Wednesday, May 03, 2017 6:40 PM To: user@drill.apache.org Cc: Kothari, Maneesh <maneesh.koth...@accenture.com> Subject: RE: Drill query are stuck in ENQUEUED mode IN false state, over the period of time queries were getting stuck in ENQUEUED state because of which I had turned this to TRUE and now they are even getting stuck with this property as TRUE. -----Original Message----- From: Khurram Faraaz [mailto:kfar...@mapr.com] Sent: Wednesday, May 03, 2017 6:35 PM To: user@drill.apache.org Cc: Kothari, Maneesh <maneesh.koth...@accenture.com> Subject: Re: Drill query are stuck in ENQUEUED mode Does your query execute and complete when you set exec.queue.enable = false ? The default is to set exec.queue.enable to false. Thanks, Khurram ________________________________ From: jasbir.s...@accenture.com <jasbir.s...@accenture.com> Sent: Wednesday, May 3, 2017 5:58:48 PM To: user@drill.apache.org Cc: maneesh.koth...@accenture.com Subject: Drill query are stuck in ENQUEUED mode Hi, I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL. I am using exec.queue.enable = true and have default settings for the rest. Can you help me out in this. Regards, Jasbir Singh ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com<http://www.accenture.com>
