+1 Regarding IMPALA-9690 if someone has the time to clean up the old conditional cpu logic it would be great, but if the ARM folks still need some of that logic it's not critical enough to block, just a very nice to have?
-Shant On Sun, Apr 4, 2021 at 1:09 AM Tim Armstrong <tim.g.armstr...@gmail.com> wrote: > > I agree that it would be good to get this done even if it's not perfect, > there's so much good stuff. > > On Fri, 2 Apr 2021 at 17:40, Joe McDonnell <joemcdonn...@cloudera.com> > wrote: > > > I agree that we should wrap up Impala 4. A large amount of good work has > > gone in and it belongs in a release. > > > > We need to decide which breaking changes are truly blockers for an Impala 4 > > release. My feeling is that we can't hold the release for compatibility > > breaking changes unless someone signs up to do those changes. Breaking > > compatibility is useful sometimes, but it can't be an indefinite hold on > > releases. I think a release discussion thread is a good way to stimulate > > this discussion. > > > > Apart from the two JIRAs you listed, one other change mentioned in the > > original email is switching to use the new on-demand metadata by default. > > > > Thanks, > > Joe > > > > On Fri, Apr 2, 2021 at 5:22 PM Quanlong Huang <huangquanl...@gmail.com> > > wrote: > > > > > Sure. If there are no objections, I'll raise a discussion thread for the > > > 4.0 release. There are still some unresolved breaking changes, e.g. > > > > > > - IMPALA-2210: Make Parquet the default file format > > > - IMPALA-9690: Bump minimum x86-64 CPU requirements > > > > > > Thanks > > > Quanlong > > > > > > On Tue, Mar 30, 2021 at 11:13 PM Jim Apple <jbap...@apache.org> wrote: > > > > > > > Thanks for bringing this up again, Quanlong! I would love to see a 4.0 > > > > release soon so that 4.1 releases and what not can be prepared. Already > > > the > > > > changlist from the 3.x line is going to be enormous and could be hard > > for > > > > users to digest. > > > > > > > > On Mon, Mar 29, 2021 at 7:22 AM Quanlong Huang < > > huangquanl...@gmail.com> > > > > wrote: > > > > > > > > > Reviving this thread. I think it's time to create the 4.0.0 branch > > and > > > > > prepare for the release now. Any breaking change is landing and we > > > should > > > > > wait? > > > > > > > > > > Thanks, > > > > > Quanlong > > > > > > > > > > On Thu, Jun 18, 2020 at 5:47 AM Tim Armstrong < > > tarmstr...@cloudera.com > > > > > > > > > wrote: > > > > > > > > > > > Another thing that we're looking into is changing the encoded > > runtime > > > > > > profile representation to be more efficient - see > > > > > > https://issues.apache.org/jira/browse/IMPALA-9378. So that might > > be > > > a > > > > > > default we will try to change in Impala 4.0. > > > > > > > > > > > > On Fri, Apr 24, 2020 at 12:37 PM Tim Armstrong < > > > > tarmstr...@cloudera.com> > > > > > > wrote: > > > > > > > > > > > > > An addendum to this - we're also considering whether to increase > > > the > > > > > > > minimum CPU version so that we can simplify development and focus > > > > more > > > > > on > > > > > > > optimising for the most common CPUs - see > > > > > > > https://issues.apache.org/jira/browse/IMPALA-9690 > > > > > > > > > > > > > > Running on commodity hardware is important and we want people to > > be > > > > > able > > > > > > > to develop on whatever machine they have access to. At most we're > > > > > likely > > > > > > > going to require AVX2 support, which has been supported by the > > vast > > > > > > > majority of CPUs for the better part of a decade. > > > > > > > > > > > > > > On Tue, Apr 7, 2020 at 11:50 AM Joe McDonnell < > > > > > joemcdonn...@cloudera.com > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Impala is starting development on Impala 4.0. There are many > > > > exciting > > > > > > >> projects underway including full support for Hive ACID tables, > > > > > improved > > > > > > >> multithreading support, and fault tolerance improvements, but > > > Impala > > > > > 4.0 > > > > > > >> also serves as a time to alter behavior and platform support. > > Here > > > > is > > > > > > the > > > > > > >> current list of notable breaking changes that are planned in > > > Impala > > > > > 4.0: > > > > > > >> > > > > > > >> Remove support for older operating systems: > > > > > > >> > > > > > > >> Maintaining support for older operating systems requires a > > > > > considerable > > > > > > >> amount of effort, especially with changing security > > requirements. > > > > Each > > > > > > >> extra operating system consumes valuable effort and can conflict > > > > with > > > > > > >> support for newer operating systems, so we have decided to > > remove > > > > > > support > > > > > > >> for some older operating systems. Specifically, we plan to drop > > > > > support > > > > > > for: > > > > > > >> > > > > > > >> - > > > > > > >> > > > > > > >> Centos 6 > > > > > > >> - > > > > > > >> > > > > > > >> Ubuntu 14 > > > > > > >> - > > > > > > >> > > > > > > >> Debian 8 > > > > > > >> > > > > > > >> Each of these is approaching or past its end of life. Since > > > Centos6 > > > > > was > > > > > > >> the only operating system still needing Python 2.6 and Python > > 2.6 > > > is > > > > > > long > > > > > > >> past its end of life, Python 2.6 will no longer be supported. In > > > > > > addition, > > > > > > >> Centos 7 support will be focused on newer versions such as 7.5 > > and > > > > > > above. > > > > > > >> > > > > > > >> Remove support for Sentry: > > > > > > >> > > > > > > >> Over the past year and a half, Impala's Ranger authorization > > > > > > >> functionality has achieved parity and surpassed the existing > > > Sentry > > > > > > >> authorization functionality. Impala's Sentry support requires > > > > ongoing > > > > > > >> effort to test and maintain, including maintenance on Sentry > > > itself > > > > to > > > > > > >> address security updates. Given the diminished activity in the > > > > Sentry > > > > > > >> community, there is no timeline for addressing these security > > > > updates. > > > > > > >> Unless there is a maintainer for Sentry support, Impala plans to > > > > focus > > > > > > its > > > > > > >> efforts on Ranger for its authorization system and drop Sentry. > > > > > > >> > > > > > > >> Remove support for Impala-lzo: > > > > > > >> > > > > > > >> Impala-lzo provides code to allow Impala to read the LZO > > > compressed > > > > > > >> tables. LZO is GPL licensed, which is why this support is not > > > > included > > > > > > >> directly. The Impala-lzo code interacts with internal Impala > > code > > > > at a > > > > > > >> level that is error prone and intricate. Given the low adoption > > of > > > > LZO > > > > > > and > > > > > > >> the other compression options available, Impala plans to remove > > > > > > Impala-lzo > > > > > > >> support along with the low level interface it used. > > > > > > >> > > > > > > >> Deprecations: > > > > > > >> > > > > > > >> In addition, we also plan to deprecate several existing features > > > so > > > > > that > > > > > > >> they can be removed in a future release. Here is a summary of > > the > > > > most > > > > > > >> notable ones: > > > > > > >> > > > > > > >> - > > > > > > >> > > > > > > >> Impala will default to using on demand metadata management > > > (i.e. > > > > > the > > > > > > >> local catalog implementation) as described in this > > > documentation: > > > > > > >> > > > > > http://impala.apache.org/docs/build/html/topics/impala_metadata.html > > > > > > >> The old metadata system is now deprecated and may be removed > > > in a > > > > > > >> future release. > > > > > > >> - > > > > > > >> > > > > > > >> Impala is deprecating the Beeswax client protocol (i.e. > > clients > > > > > that > > > > > > >> connect via beeswax_port) in favor of the HiveServer2 client > > > > > > protocol. > > > > > > >> - > > > > > > >> > > > > > > >> Impala is deprecating the old decimal_v2=false behavior from > > > > Impala > > > > > > >> 2.x. The decimal_v2 query option has defaulted to true since > > > > Impala > > > > > > 3. > > > > > > >> > > > > > > >> > > > > > > >> Impala is always open to new developers, and we welcome feedback > > > on > > > > > > these > > > > > > >> plans. Further discussion of other changes and deprecations is > > > > ongoing > > > > > > at > > > > > > >> dev@impala.apache.org. > > > > > > >> > > > > > > >> > > > > > > >> Thanks, > > > > > > >> > > > > > > >> Joe McDonnell > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > >
