I didn't have any specifics in mind when I wrote there are good commits on master since May 1. My principle was only that it would be nice to release what's latest, since some time has passed since 4.0 was branched.
Now that I looked at the commit list, maybe we could consider adding the followings at least: - IMPALA-10683: Skip test 'test_double_precision' for non-HDFS test env - IMPALA-9690: Set minimum CPU requirement to AVX2 for x86_64 - IMPALA-10413: fix impalad crashes when canceling the retrying query - IMPALA-5121: Fix AVG() on timestamp col with use_local_tz_for_unix_timestamp_conversions - IMPALA-10704: Fix retried query id not being unregistered when retry fails IMPALA-9690 <https://issues.apache.org/jira/browse/IMPALA-9690> and IMPALA-5121 <https://issues.apache.org/jira/browse/IMPALA-5121> are even targeted for 4.0 in Jira. IMPALA-10413 <https://issues.apache.org/jira/browse/IMPALA-10413> is targeted for 4.0, but the Jira is not resolved yet. A patch has been already merged, so I asked at the Jira ticket whether we can resolve it. IMPALA-10704 <https://issues.apache.org/jira/browse/IMPALA-10704> is targeted for 4.1, but fixes an issue introduced by IMPALA-10413. Thanks, Zoltan On Fri, Jun 4, 2021 at 5:06 AM Jim Apple <jbap...@apache.org> wrote: > Once we know the answer to that last question, I'd be interested in reading > more about your vision for what should go in 4.0, Quanlong -- > i.e."everything committed on June 2 except patches X and Y". > > On Thu, Jun 3, 2021 at 2:10 AM Quanlong Huang <huangquanl...@gmail.com> > wrote: > > > Sorry to be late here! I'm -1 for merging HEAD into 4.0. Because it will > > bump the CDP_BUILD_NUMBER again which increases the difficulty for 3.x > > users (who may still be using Hive 2.x or 3.x) to adopt Impala-4.0. There > > are two new commits in HEAD that bump the CDP_BUILD_NUMBER. > > > > We may have already depended on some Hive 4.0 APIs. But those features > may > > not be needed by 3.x users. They can revert some changes to make > Impala-4.0 > > compatible with older versions of Hive. > > The lower GBN we have, the easier it is for them to upgrade. > > > > Does those good commits of HEAD depend on bumping the GBN? If not, we can > > just pick them to the 4.0 branch. > > > > Thanks, > > Quanlong > > > > On Wed, May 26, 2021 at 8:52 PM Zoltán Borók-Nagy <borokna...@apache.org > > > > wrote: > > > > > There's been a handful of good commits since May 1, so +1 for merging > > HEAD > > > into 4.0. > > > > > > Cheers, > > > Zoltan > > > > > > On Wed, May 26, 2021 at 2:11 PM Jim Apple <jbap...@apache.org> wrote: > > > > > > > We're only waiting on one patch for the docs, but since I cut the 4.0 > > > > branch on May 1, a number of things have landed. Maybe it would be > > best > > > to > > > > merge HEAD into 4.0.0 once the Sentry doc change lands, thereby > > including > > > > everything we've talked about cherrypicking -- including 9690. > > > > > > > > Thoughts? > > > > > > > > On Wed, May 26, 2021 at 12:40 AM Quanlong Huang < > > huangquanl...@gmail.com > > > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I see IMPALA-9690 is done but still missing in branch-4.0.0 and I > > think > > > > we > > > > > should pick it > > > > > > > > > > - c10e7c9 IMPALA-9690: Set minimum CPU requirement to AVX2 for > > > x86_64 > > > > > > > > > > I'd be good to also carry on these fixes for query retry: > > > > > > > > > > - 42684b4 IMPALA-10413: fix impalad crashes when canceling the > > > > retrying > > > > > query > > > > > - d111443 IMPALA-10704: Fix retried query id not being > > unregistered > > > > when > > > > > retry fails > > > > > > > > > > If you are ok, I can push them to branch-4.0.0. > > > > > > > > > > Thanks, > > > > > Quanlong > > > > > > > > > > On Sun, May 16, 2021 at 6:02 AM Jim Apple <jbap...@apache.org> > > wrote: > > > > > > > > > > > Sounds good! > > > > > > > > > > > > On Sat, May 15, 2021 at 1:29 PM Joe McDonnell < > > > > joemcdonn...@cloudera.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Hi Jim, > > > > > > > > > > > > > > I talked with the developer for the Sentry docs bug this past > > week, > > > > and > > > > > > we > > > > > > > went through the list of changes needed for that. The developer > > is > > > > > hoping > > > > > > > to work on it next week. > > > > > > > > > > > > > > If there is any further delay, I'm open to options of working > > > around > > > > > it. > > > > > > > > > > > > > > Thanks, > > > > > > > Joe > > > > > > > > > > > > > > On Sat, May 15, 2021 at 12:43 PM Jim Apple <jbap...@apache.org > > > > > > wrote: > > > > > > > > > > > > > > > This Sentry one I'm concerned about. The ticket has been open > > > for a > > > > > > year. > > > > > > > > We could do a 4.0 release without the docs, or we could maybe > > > put a > > > > > > note > > > > > > > on > > > > > > > > the docs that Sentry support is no longer available. > > > > > > > > > > > > > > > > Any other ideas on how to tackle this while the ticket waits > > for > > > a > > > > > > patch? > > > > > > > > > > > > > > > > On Thu, May 6, 2021 at 8:36 PM Joe McDonnell < > > > > > > joemcdonn...@cloudera.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > I agree that it is a good idea for us to get going on the > 4.0 > > > > > > release. > > > > > > > > > > > > > > > > > > I'm working on a change to fix IMPALA-9690. I will post an > > > > updated > > > > > > > review > > > > > > > > > once a test run completes. If everything is looking good, I > > > would > > > > > > like > > > > > > > to > > > > > > > > > get this in the 4.0 release. > > > > > > > > > > > > > > > > > > There is also one documentation fix that I think belongs in > > the > > > > > > Impala > > > > > > > > 4.0 > > > > > > > > > release: > > > > > > > > > https://issues.apache.org/jira/browse/IMPALA-9770 - Remove > > > > Sentry > > > > > > > > > references > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Joe > > > > > > > > > > > > > > > > > > On Thu, May 6, 2021 at 7:55 PM Jim Apple < > apa...@jbapple.com > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Sure thing! Let us know if you find owners for anything > > that > > > > > > someone > > > > > > > > > wants > > > > > > > > > > to champion as a 4.0 blocker. > > > > > > > > > > > > > > > > > > > > On Thu, May 6, 2021 at 7:08 PM Quanlong Huang < > > > > > > > huangquanl...@gmail.com > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Thank Jim! I planned to go through all the JIRAs target > > at > > > > 4.0: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IMPALA-10690?jql=project%20%3D%20IMPALA%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%22%20%3D%20%22Impala%204.0%22 > > > > > > > > > > > > > > > > > > > > > > But due to some other urgent works and the recent > > holiday, > > > my > > > > > > > > progress > > > > > > > > > is > > > > > > > > > > > slow. Anyway, I think it's time to branch out. We've > been > > > > > waiting > > > > > > > too > > > > > > > > > > long. > > > > > > > > > > > Thanks for creating the branch. > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > Quanlong > > > > > > > > > > > > > > > > > > > > > > On Fri, May 7, 2021 at 3:29 AM Jim Apple < > > > apa...@jbapple.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > I've created a 4.0 branch (with all changes through > the > > > end > > > > > of > > > > > > > > April) > > > > > > > > > > and > > > > > > > > > > > > started testing a release candidate. Everyone who > wants > > > to > > > > > > vote, > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release > > > > > > > > > is a > > > > > > > > > > > > guide to both releasing and voting (at the bottom). > > I'll > > > > let > > > > > > you > > > > > > > > all > > > > > > > > > > know > > > > > > > > > > > > when I have something to vote on. > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > Jim > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 23, 2021 at 12:49 AM Csaba Ringhofer < > > > > > > > > > > > csringho...@cloudera.com > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > About IMPALA-9690 (AVX support): > > > > > > > > > > > > > My preferred solution would be to deprecate support > > for > > > > x64 > > > > > > > > without > > > > > > > > > > > AVX2 > > > > > > > > > > > > in > > > > > > > > > > > > > 4.0, but not start removing the related logic yet. > > > > > > > > > > > > > - We could even add a DCHECK + flag to crash by > > default > > > > if > > > > > no > > > > > > > > AVX2 > > > > > > > > > is > > > > > > > > > > > > > detected, and a message that points them to Impala > > > > mailing > > > > > > > > > > > lists/related > > > > > > > > > > > > > Jira. > > > > > > > > > > > > > - After 4.0 is released people would be free to > > > implement > > > > > > > things > > > > > > > > > that > > > > > > > > > > > > > assume AVX2 support on x64, for example compiling > > > > > Impala/the > > > > > > > > whole > > > > > > > > > > > > > toolchain with -mavx2. I think that currently > people > > > are > > > > > held > > > > > > > > back > > > > > > > > > > from > > > > > > > > > > > > > doing this - why put effort in doing+benchmarking a > > > > change > > > > > if > > > > > > > you > > > > > > > > > > can't > > > > > > > > > > > > > push it to master. > > > > > > > > > > > > > - If a user appears with a valid use case that > needs > > > x64 > > > > > > > without > > > > > > > > > AVX2 > > > > > > > > > > > > then > > > > > > > > > > > > > we can revise our plans, but we shouldn't ignore > low > > > > > hanging > > > > > > > > fruits > > > > > > > > > > for > > > > > > > > > > > > the > > > > > > > > > > > > > sake of imaginary users :) > > > > > > > > > > > > > > > > > > > > > > > > > > -Csaba > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 11, 2021 at 5:52 PM Shant Hovsepian < > > > > > > > > > > > > sh...@superdupershant.com > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regarding IMPALA-9690 if someone has the time to > > > clean > > > > up > > > > > > the > > > > > > > > old > > > > > > > > > > > > > > conditional cpu logic it would be great, but if > the > > > ARM > > > > > > folks > > > > > > > > > still > > > > > > > > > > > > > > need some of that logic it's not critical enough > to > > > > > block, > > > > > > > > just a > > > > > > > > > > > very > > > > > > > > > > > > > > nice to have? > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Shant > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 4, 2021 at 1:09 AM Tim Armstrong < > > > > > > > > > > > > tim.g.armstr...@gmail.com> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree that it would be good to get this done > > even > > > > if > > > > > > it's > > > > > > > > not > > > > > > > > > > > > > perfect, > > > > > > > > > > > > > > > there's so much good stuff. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 2 Apr 2021 at 17:40, Joe McDonnell < > > > > > > > > > > > > joemcdonn...@cloudera.com> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree that we should wrap up Impala 4. A > > large > > > > > amount > > > > > > > of > > > > > > > > > good > > > > > > > > > > > > work > > > > > > > > > > > > > > has > > > > > > > > > > > > > > > > gone in and it belongs in a release. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We need to decide which breaking changes are > > > truly > > > > > > > blockers > > > > > > > > > for > > > > > > > > > > > an > > > > > > > > > > > > > > Impala 4 > > > > > > > > > > > > > > > > release. My feeling is that we can't hold the > > > > release > > > > > > for > > > > > > > > > > > > > compatibility > > > > > > > > > > > > > > > > breaking changes unless someone signs up to > do > > > > those > > > > > > > > changes. > > > > > > > > > > > > > Breaking > > > > > > > > > > > > > > > > compatibility is useful sometimes, but it > can't > > > be > > > > an > > > > > > > > > > indefinite > > > > > > > > > > > > hold > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > releases. I think a release discussion thread > > is > > > a > > > > > good > > > > > > > way > > > > > > > > > to > > > > > > > > > > > > > > stimulate > > > > > > > > > > > > > > > > this discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apart from the two JIRAs you listed, one > other > > > > change > > > > > > > > > mentioned > > > > > > > > > > > in > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > original email is switching to use the new > > > > on-demand > > > > > > > > metadata > > > > > > > > > > by > > > > > > > > > > > > > > default. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Joe > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 2, 2021 at 5:22 PM Quanlong > Huang < > > > > > > > > > > > > > huangquanl...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sure. If there are no objections, I'll > raise > > a > > > > > > > discussion > > > > > > > > > > > thread > > > > > > > > > > > > > for > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > 4.0 release. There are still some > unresolved > > > > > breaking > > > > > > > > > > changes, > > > > > > > > > > > > e.g. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > - IMPALA-2210: Make Parquet the default > > file > > > > > > format > > > > > > > > > > > > > > > > > - IMPALA-9690: Bump minimum x86-64 CPU > > > > > > requirements > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > Quanlong > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 30, 2021 at 11:13 PM Jim Apple > < > > > > > > > > > > jbap...@apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for bringing this up again, > > Quanlong! > > > I > > > > > > would > > > > > > > > love > > > > > > > > > > to > > > > > > > > > > > > see > > > > > > > > > > > > > a > > > > > > > > > > > > > > 4.0 > > > > > > > > > > > > > > > > > > release soon so that 4.1 releases and > what > > > not > > > > > can > > > > > > be > > > > > > > > > > > prepared. > > > > > > > > > > > > > > Already > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > changlist from the 3.x line is going to > be > > > > > enormous > > > > > > > and > > > > > > > > > > could > > > > > > > > > > > > be > > > > > > > > > > > > > > hard > > > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > > users to digest. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 29, 2021 at 7:22 AM Quanlong > > > Huang > > > > < > > > > > > > > > > > > > > > > huangquanl...@gmail.com> > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Reviving this thread. I think it's time > > to > > > > > create > > > > > > > the > > > > > > > > > > 4.0.0 > > > > > > > > > > > > > > branch > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > prepare for the release now. Any > breaking > > > > > change > > > > > > is > > > > > > > > > > landing > > > > > > > > > > > > and > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > > > wait? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > Quanlong > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 18, 2020 at 5:47 AM Tim > > > > Armstrong < > > > > > > > > > > > > > > > > tarmstr...@cloudera.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Another thing that we're looking into > > is > > > > > > changing > > > > > > > > the > > > > > > > > > > > > encoded > > > > > > > > > > > > > > > > runtime > > > > > > > > > > > > > > > > > > > > profile representation to be more > > > > efficient - > > > > > > see > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IMPALA-9378. > > > > > > > > > So > > > > > > > > > > > that > > > > > > > > > > > > > > might > > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > > > > default we will try to change in > Impala > > > > 4.0. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 24, 2020 at 12:37 PM Tim > > > > > Armstrong > > > > > > < > > > > > > > > > > > > > > > > > > tarmstr...@cloudera.com> > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > An addendum to this - we're also > > > > > considering > > > > > > > > > whether > > > > > > > > > > to > > > > > > > > > > > > > > increase > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > minimum CPU version so that we can > > > > simplify > > > > > > > > > > development > > > > > > > > > > > > and > > > > > > > > > > > > > > focus > > > > > > > > > > > > > > > > > > more > > > > > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > > > > > optimising for the most common > CPUs - > > > see > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IMPALA-9690 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Running on commodity hardware is > > > > important > > > > > > and > > > > > > > we > > > > > > > > > > want > > > > > > > > > > > > > > people to > > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > > > able > > > > > > > > > > > > > > > > > > > > > to develop on whatever machine they > > > have > > > > > > access > > > > > > > > to. > > > > > > > > > > At > > > > > > > > > > > > most > > > > > > > > > > > > > > we're > > > > > > > > > > > > > > > > > > > likely > > > > > > > > > > > > > > > > > > > > > going to require AVX2 support, > which > > > has > > > > > been > > > > > > > > > > supported > > > > > > > > > > > > by > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > vast > > > > > > > > > > > > > > > > > > > > > majority of CPUs for the better > part > > > of a > > > > > > > decade. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 7, 2020 at 11:50 AM Joe > > > > > > McDonnell < > > > > > > > > > > > > > > > > > > > joemcdonn...@cloudera.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> Impala is starting development on > > > Impala > > > > > > 4.0. > > > > > > > > > There > > > > > > > > > > > are > > > > > > > > > > > > > many > > > > > > > > > > > > > > > > > > exciting > > > > > > > > > > > > > > > > > > > > >> projects underway including full > > > support > > > > > for > > > > > > > > Hive > > > > > > > > > > ACID > > > > > > > > > > > > > > tables, > > > > > > > > > > > > > > > > > > > improved > > > > > > > > > > > > > > > > > > > > >> multithreading support, and fault > > > > > tolerance > > > > > > > > > > > > improvements, > > > > > > > > > > > > > > but > > > > > > > > > > > > > > > > > Impala > > > > > > > > > > > > > > > > > > > 4.0 > > > > > > > > > > > > > > > > > > > > >> also serves as a time to alter > > > behavior > > > > > and > > > > > > > > > platform > > > > > > > > > > > > > > support. > > > > > > > > > > > > > > > > Here > > > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > >> current list of notable breaking > > > changes > > > > > > that > > > > > > > > are > > > > > > > > > > > > planned > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > Impala > > > > > > > > > > > > > > > > > > > 4.0: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Remove support for older operating > > > > > systems: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Maintaining support for older > > > operating > > > > > > > systems > > > > > > > > > > > > requires a > > > > > > > > > > > > > > > > > > > considerable > > > > > > > > > > > > > > > > > > > > >> amount of effort, especially with > > > > changing > > > > > > > > > security > > > > > > > > > > > > > > > > requirements. > > > > > > > > > > > > > > > > > > Each > > > > > > > > > > > > > > > > > > > > >> extra operating system consumes > > > valuable > > > > > > > effort > > > > > > > > > and > > > > > > > > > > > can > > > > > > > > > > > > > > conflict > > > > > > > > > > > > > > > > > > with > > > > > > > > > > > > > > > > > > > > >> support for newer operating > systems, > > > so > > > > we > > > > > > > have > > > > > > > > > > > decided > > > > > > > > > > > > to > > > > > > > > > > > > > > > > remove > > > > > > > > > > > > > > > > > > > > support > > > > > > > > > > > > > > > > > > > > >> for some older operating systems. > > > > > > > Specifically, > > > > > > > > we > > > > > > > > > > > plan > > > > > > > > > > > > to > > > > > > > > > > > > > > drop > > > > > > > > > > > > > > > > > > > support > > > > > > > > > > > > > > > > > > > > for: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> - > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Centos 6 > > > > > > > > > > > > > > > > > > > > >> - > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Ubuntu 14 > > > > > > > > > > > > > > > > > > > > >> - > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Debian 8 > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Each of these is approaching or > past > > > its > > > > > end > > > > > > > of > > > > > > > > > > life. > > > > > > > > > > > > > Since > > > > > > > > > > > > > > > > > Centos6 > > > > > > > > > > > > > > > > > > > was > > > > > > > > > > > > > > > > > > > > >> the only operating system still > > > needing > > > > > > Python > > > > > > > > 2.6 > > > > > > > > > > and > > > > > > > > > > > > > > Python > > > > > > > > > > > > > > > > 2.6 > > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > > > > long > > > > > > > > > > > > > > > > > > > > >> past its end of life, Python 2.6 > > will > > > no > > > > > > > longer > > > > > > > > be > > > > > > > > > > > > > > supported. In > > > > > > > > > > > > > > > > > > > > addition, > > > > > > > > > > > > > > > > > > > > >> Centos 7 support will be focused > on > > > > newer > > > > > > > > versions > > > > > > > > > > > such > > > > > > > > > > > > as > > > > > > > > > > > > > > 7.5 > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > > above. > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Remove support for Sentry: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Over the past year and a half, > > > Impala's > > > > > > Ranger > > > > > > > > > > > > > authorization > > > > > > > > > > > > > > > > > > > > >> functionality has achieved parity > > and > > > > > > > surpassed > > > > > > > > > the > > > > > > > > > > > > > existing > > > > > > > > > > > > > > > > > Sentry > > > > > > > > > > > > > > > > > > > > >> authorization functionality. > > Impala's > > > > > Sentry > > > > > > > > > support > > > > > > > > > > > > > > requires > > > > > > > > > > > > > > > > > > ongoing > > > > > > > > > > > > > > > > > > > > >> effort to test and maintain, > > including > > > > > > > > maintenance > > > > > > > > > > on > > > > > > > > > > > > > Sentry > > > > > > > > > > > > > > > > > itself > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > >> address security updates. Given > the > > > > > > diminished > > > > > > > > > > > activity > > > > > > > > > > > > in > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > Sentry > > > > > > > > > > > > > > > > > > > > >> community, there is no timeline > for > > > > > > addressing > > > > > > > > > these > > > > > > > > > > > > > > security > > > > > > > > > > > > > > > > > > updates. > > > > > > > > > > > > > > > > > > > > >> Unless there is a maintainer for > > > Sentry > > > > > > > support, > > > > > > > > > > > Impala > > > > > > > > > > > > > > plans to > > > > > > > > > > > > > > > > > > focus > > > > > > > > > > > > > > > > > > > > its > > > > > > > > > > > > > > > > > > > > >> efforts on Ranger for its > > > authorization > > > > > > system > > > > > > > > and > > > > > > > > > > > drop > > > > > > > > > > > > > > Sentry. > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Remove support for Impala-lzo: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Impala-lzo provides code to allow > > > Impala > > > > > to > > > > > > > read > > > > > > > > > the > > > > > > > > > > > LZO > > > > > > > > > > > > > > > > > compressed > > > > > > > > > > > > > > > > > > > > >> tables. LZO is GPL licensed, which > > is > > > > why > > > > > > this > > > > > > > > > > support > > > > > > > > > > > > is > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > included > > > > > > > > > > > > > > > > > > > > >> directly. The Impala-lzo code > > > interacts > > > > > with > > > > > > > > > > internal > > > > > > > > > > > > > Impala > > > > > > > > > > > > > > > > code > > > > > > > > > > > > > > > > > > at a > > > > > > > > > > > > > > > > > > > > >> level that is error prone and > > > intricate. > > > > > > Given > > > > > > > > the > > > > > > > > > > low > > > > > > > > > > > > > > adoption > > > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > > > LZO > > > > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > > >> the other compression options > > > available, > > > > > > > Impala > > > > > > > > > > plans > > > > > > > > > > > to > > > > > > > > > > > > > > remove > > > > > > > > > > > > > > > > > > > > Impala-lzo > > > > > > > > > > > > > > > > > > > > >> support along with the low level > > > > interface > > > > > > it > > > > > > > > > used. > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Deprecations: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> In addition, we also plan to > > deprecate > > > > > > several > > > > > > > > > > > existing > > > > > > > > > > > > > > features > > > > > > > > > > > > > > > > > so > > > > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > > > >> they can be removed in a future > > > release. > > > > > > Here > > > > > > > > is a > > > > > > > > > > > > summary > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > most > > > > > > > > > > > > > > > > > > > > >> notable ones: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> - > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Impala will default to using on > > > > demand > > > > > > > > metadata > > > > > > > > > > > > > > management > > > > > > > > > > > > > > > > > (i.e. > > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > >> local catalog implementation) > as > > > > > > described > > > > > > > in > > > > > > > > > > this > > > > > > > > > > > > > > > > > documentation: > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://impala.apache.org/docs/build/html/topics/impala_metadata.html > > > > > > > > > > > > > > > > > > > > >> The old metadata system is now > > > > > deprecated > > > > > > > and > > > > > > > > > may > > > > > > > > > > > be > > > > > > > > > > > > > > removed > > > > > > > > > > > > > > > > > in a > > > > > > > > > > > > > > > > > > > > >> future release. > > > > > > > > > > > > > > > > > > > > >> - > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Impala is deprecating the > Beeswax > > > > > client > > > > > > > > > protocol > > > > > > > > > > > > (i.e. > > > > > > > > > > > > > > > > clients > > > > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > > > >> connect via beeswax_port) in > > favor > > > of > > > > > the > > > > > > > > > > > HiveServer2 > > > > > > > > > > > > > > client > > > > > > > > > > > > > > > > > > > > protocol. > > > > > > > > > > > > > > > > > > > > >> - > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Impala is deprecating the old > > > > > > > > decimal_v2=false > > > > > > > > > > > > behavior > > > > > > > > > > > > > > from > > > > > > > > > > > > > > > > > > Impala > > > > > > > > > > > > > > > > > > > > >> 2.x. The decimal_v2 query > option > > > has > > > > > > > > defaulted > > > > > > > > > to > > > > > > > > > > > > true > > > > > > > > > > > > > > since > > > > > > > > > > > > > > > > > > Impala > > > > > > > > > > > > > > > > > > > > 3. > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Impala is always open to new > > > developers, > > > > > and > > > > > > > we > > > > > > > > > > > welcome > > > > > > > > > > > > > > feedback > > > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > > > > these > > > > > > > > > > > > > > > > > > > > >> plans. Further discussion of other > > > > changes > > > > > > and > > > > > > > > > > > > > deprecations > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > > ongoing > > > > > > > > > > > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > > >> dev@impala.apache.org. > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Thanks, > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> Joe McDonnell > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
