During implementing late limit 0 optimization, Bohdan has found one more regression after Hash Join spill to disk. https://issues.apache.org/jira/browse/DRILL-6606 Boaz please take a look.
Kind regards, Arina On Fri, Jul 13, 2018 at 4:34 AM Boaz Ben-Zvi <[email protected]> wrote: > We are getting close to a Release Candidate, though some issues are > still pending, and we need to make decisions soon. > > Soliciting opinions -- which of the following issues should be > considered a RELEASE BLOCKER for 1.14: > > ===== OPEN ====== > > OPEN - DRILL-6453 : TPCDS query 72 is Hanging (on a cluster) (( Boaz, > Salim )) > > We still do not have a lead on the cause, nor a work around to make > this query run. > > OPEN - DRILL-6475: Query with UNNEST causes a Null Pointer . (( Hanumath > )) > > OPEN - DRILL-5495: convert_from causes ArrayIndexOutOfBounds exception. > (( Vitalii )) > > ==== In Review ======= > > DRILL-6589: Push Transitive Closure generated predicates past aggregates > / projects ((Gautam / Vitalii)) > > DRILL-6588: System table columns incorrectly marked as non-nullable > ((Kunal / Aman)) > > DRILL-6542: (May be Ready2Commit soon) IndexOutOfBounds exception for > multilevel lateral ((Sorabh / Parth)) > > DRILL-6517: (May be Ready2Commit soon) IllegalState exception in > Hash-Join ((Boaz / Padma, Tim)) > > DRILL-6496: VectorUtil.showVectorAccessibleContent does not log vector > content ((Tim / Volodymyr)) > > DRILL-6410: Memory Leak in Parquet Reader during cancellation ((Vlad / > Parth)) > > DRILL-6179: Added pcapng-format support ((Vlad / Paul)) > > DRILL-5796: Filter pruning for multi rowgroup parquet file ((Jean-Blas / > Arina)) > > DRILL-5365: FileNotFoundException when reading a parquet file ((Tim / > Vitalii)) > > ====================== > > Thanks, > > -- Boaz > p.s. > There's a batch commit in process now with some of the PRs listed in > the prior email. > > On 7/9/18 9:53 PM, Boaz Ben-Zvi wrote: > > We are making progress towards 1.14. > > > > Let's aim for a Release Candidate branch off on Thursday (July 12) !!! > > > > Below are the unfinished cases; can most be completed and checked in > > by 7/12 ? > > > > (( Relevant people: > > > > Abhishek, Arina, Boaz, Charles, Hanumath, Jean-Blas, Karthik, Kunal, > > > > Parth, Paul, Salim, Sorabh, Tim, Vitalii, Vlad, Volodymyr )) > > > > ====================== > > > > Open/blocker - DRILL-6453 + DRILL-6517: > > Two issues - Parquet Scanner (?) not setting container's record num > > (to zero), and a hang following this failure. > > Currently testing a fix / workaround ((Boaz)) > > > > In Progress - DRILL-6104: Generic Logfile Format Plugin ((Charles + > > Paul -- can you be done by 7/12 ?)) > > > > PR - DRILL-5999 (DRILL-6516): Support for EMIT outcome in Streaming > > Agg ((Parth + Boaz reviewing)) > > > > Open - DRILL-6542: Index out of bounds ((Sorabh)) > > > > Open - DRILL-6475: Unnest Null fieldId pointer ((Hanumath)) > > > > ==== The following PRs are still waiting for reviews ==== > > > > DRILL-6583: UI usability issue ((Kunal / Sorabh)) > > > > DRILL-6579: Add sanity checks to the Parquet Reader ((Salim / Vlad + > > Boaz)) > > > > DRILL-6578: handle query cancellation in Parquet Reader ((Salim / Vlad > > + Boaz)) > > > > DRILL-6560: Allow options for controlling the batch size per operator > > ((Salim / Karthik)) > > > > DRILL-6559: Travis timing out ((Vitalii / Tim)) > > > > DRILL-6496: VectorUtil.showVectorAccessibleContent does not log vector > > content ((Tim / Volodymyr)) > > > > DRILL-6410: Memory Leak in Parquet Reader during cancellation ((Vlad / > > Parth)) > > > > DRILL-6346: Create an Official Drill Docker Container ((Abhishek / Tim)) > > > > DRILL-6179: Added pcapng-format support ((Vlad / Paul)) > > > > DRILL-5796: Filter pruning for multi rowgroup parquet file ((Jean-Blas > > / Arina)) > > > > DRILL-5365: FileNotFoundException when reading a parquet file ((Tim / > > Vitalii)) > > > > ================================== > > > > Thanks, > > > > Boaz > > > > On 7/6/18 2:51 PM, Pritesh Maker wrote: > >> Here is the release 1.14 dashboard > >> ( > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_secure_Dashboard.jspa-3FselectPageId-3D12332463&d=DwIGaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=7lXQnf0aC8VQ0iMXwVgNHw&m=V0b4-BeuIMwRczzbiSXCgL7Z7f8lrmLBGH1vnSHLjB4&s=HRQU6Q4umbONtN4EqY3ryggJNEyCOghAzICypRJOels&e= > >> ) and agile board > >> ( > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_secure_RapidBoard.jspa-3FrapidView-3D185&d=DwIGaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=7lXQnf0aC8VQ0iMXwVgNHw&m=V0b4-BeuIMwRczzbiSXCgL7Z7f8lrmLBGH1vnSHLjB4&s=GKxSMl97YRnHJu-AL-A-vvRe5SXqw7vdDPDzMzj-Cj4&e= > ) > >> > >> I believe Volodymyr is targeting DRILL-6422 (Guava update) for 1.15 > >> release so it shouldn't be blocking the release. So overall, we have > >> 2 open bugs, 2 in progress bugs (+2 doc issues), and 12 in review (+1 > >> ready to commit). > >> > >> If the reviewable commits won't be ready soon, can the developers > >> please remove the 1.14 fix version for these issues. > >> > >> Pritesh > >> > >> > >> > >> > >> On 7/6/18, 11:54 AM, "Boaz Ben-Zvi" <[email protected] on behalf of > >> [email protected]> wrote: > >> > >> Current status: There's a blocker, and some work in progress > >> that will > >> stretch into next week. > >> Current detail: > >> ============================== > >> Open/blocker - DRILL-6453 + DRILL-6517: Two issues - > >> Parquet Scanner not setting record num (to zero), and a hang > >> following this failure. > >> In Progress - DRILL-6104: Generic Logfile Format Plugin > >> PR - DRILL-6422: Update Guava to 23.0 and shade it > >> PR - DRILL-5999 (DRILL-6516): Support for EMIT outcome in > >> Streaming Agg (I'm reviewing) > >> Ready2Commit: DRILL-6519: Add String Distance and > >> Phonetic Functions (Arina gave it a +1 ; is it "Ready-To-Commit" or > >> waiting for more reviews ?) > >> Committed: DRILL-6570: Mentioned as a blocker by Kunal > >> (I just merge #1354; the Jira was (mistakenly ?) marked "Resolved" so > >> it missed the batch commit). > >> Committed: DRILL-5977: predicate pushdown support > >> kafkaMsgOffset > >> Committed: DRILL-6577: Change Hash-Join default to not > >> fallback (into > >> pre-1.14 unlimited memory) > >> Committed: DRILL-6353: Upgrade Parquet MR dependencies > >> Committed: DRILL-6310: limit batch size for hash aggregate > >> =============================== > >> Thanks, > >> Boaz > >> On 7/2/18 9:51 PM, Khurram Faraaz wrote: > >> > Do we plan to fix this one too, because this is a regression > >> from Apache > >> > Drill 1.13.0. > >> > > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_DRILL-2D6453&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=EqulKDxxEDCX6zbp1AZAa1-iAPQGgCioAqgDp7DE2BU&m=9HpIZKDh_DBcx9yXCI2TtUWum8vRhdQgmw_3ljJQi1M&s=w-1C-HnlUAGRHvDFUusZ78WZrHahDka2r-F-cNb-pZA&e= > >> > > >> > On Mon, Jul 2, 2018 at 9:33 PM, Kunal Khatua > >> <[email protected]> wrote: > >> > > >> >> DRILL-6570 seems like a must-have (release blocker, IMHO). > >> >> On 7/2/2018 8:02:00 PM, Boaz Ben-Zvi <[email protected]> wrote: > >> >> Let's try to make progress on the 1.14 release, aiming for a > >> Release > >> >> Candidate towards the end of this week (a little ambitious, > >> with the > >> >> July 4th and people on vacations). > >> >> > >> >> Current Status of the previously requested Jiras: > >> >> > >> >> ============================== > >> >> > >> >> In Progress - DRILL-6104: Generic Logfile Format Plugin > >> >> > >> >> PR - DRILL-6422: Update Guava to 23.0 and shade it > >> >> > >> >> PR - DRILL-5999 (DRILL-6516): Support for EMIT outcome in > >> Streaming Agg > >> >> > >> >> Ready2Commit: DRILL-5977: predicate pushdown support > >> kafkaMsgOffset > >> >> > >> >> Ready2Commit: DRILL-6519: Add String Distance and Phonetic > >> Functions > >> >> > >> >> Ready2Commit: DRILL-6577: Change Hash-Join default to not > >> fallback (into > >> >> pre-1.14 unlimited memory) > >> >> > >> >> Committed: DRILL-6353: Upgrade Parquet MR dependencies > >> >> > >> >> Committed: DRILL-6310: limit batch size for hash aggregate > >> >> > >> >> =============================== > >> >> > >> >> And there are few more open or in a PR state. > >> >> > >> >> Lets try and most of these ready by the end of the week. > >> >> > >> >> Boaz > >> >> > >> >> > >> >> > > > >
