Another thing that we're looking into is changing the encoded runtime profile representation to be more efficient - see https://issues.apache.org/jira/browse/IMPALA-9378. So that might be a default we will try to change in Impala 4.0.
On Fri, Apr 24, 2020 at 12:37 PM Tim Armstrong <tarmstr...@cloudera.com> wrote: > An addendum to this - we're also considering whether to increase the > minimum CPU version so that we can simplify development and focus more on > optimising for the most common CPUs - see > https://issues.apache.org/jira/browse/IMPALA-9690 > > Running on commodity hardware is important and we want people to be able > to develop on whatever machine they have access to. At most we're likely > going to require AVX2 support, which has been supported by the vast > majority of CPUs for the better part of a decade. > > On Tue, Apr 7, 2020 at 11:50 AM Joe McDonnell <joemcdonn...@cloudera.com> > wrote: > >> Impala is starting development on Impala 4.0. There are many exciting >> projects underway including full support for Hive ACID tables, improved >> multithreading support, and fault tolerance improvements, but Impala 4.0 >> also serves as a time to alter behavior and platform support. Here is the >> current list of notable breaking changes that are planned in Impala 4.0: >> >> Remove support for older operating systems: >> >> Maintaining support for older operating systems requires a considerable >> amount of effort, especially with changing security requirements. Each >> extra operating system consumes valuable effort and can conflict with >> support for newer operating systems, so we have decided to remove support >> for some older operating systems. Specifically, we plan to drop support for: >> >> - >> >> Centos 6 >> - >> >> Ubuntu 14 >> - >> >> Debian 8 >> >> Each of these is approaching or past its end of life. Since Centos6 was >> the only operating system still needing Python 2.6 and Python 2.6 is long >> past its end of life, Python 2.6 will no longer be supported. In addition, >> Centos 7 support will be focused on newer versions such as 7.5 and above. >> >> Remove support for Sentry: >> >> Over the past year and a half, Impala's Ranger authorization >> functionality has achieved parity and surpassed the existing Sentry >> authorization functionality. Impala's Sentry support requires ongoing >> effort to test and maintain, including maintenance on Sentry itself to >> address security updates. Given the diminished activity in the Sentry >> community, there is no timeline for addressing these security updates. >> Unless there is a maintainer for Sentry support, Impala plans to focus its >> efforts on Ranger for its authorization system and drop Sentry. >> >> Remove support for Impala-lzo: >> >> Impala-lzo provides code to allow Impala to read the LZO compressed >> tables. LZO is GPL licensed, which is why this support is not included >> directly. The Impala-lzo code interacts with internal Impala code at a >> level that is error prone and intricate. Given the low adoption of LZO and >> the other compression options available, Impala plans to remove Impala-lzo >> support along with the low level interface it used. >> >> Deprecations: >> >> In addition, we also plan to deprecate several existing features so that >> they can be removed in a future release. Here is a summary of the most >> notable ones: >> >> - >> >> Impala will default to using on demand metadata management (i.e. the >> local catalog implementation) as described in this documentation: >> http://impala.apache.org/docs/build/html/topics/impala_metadata.html >> The old metadata system is now deprecated and may be removed in a >> future release. >> - >> >> Impala is deprecating the Beeswax client protocol (i.e. clients that >> connect via beeswax_port) in favor of the HiveServer2 client protocol. >> - >> >> Impala is deprecating the old decimal_v2=false behavior from Impala >> 2.x. The decimal_v2 query option has defaulted to true since Impala 3. >> >> >> Impala is always open to new developers, and we welcome feedback on these >> plans. Further discussion of other changes and deprecations is ongoing at >> dev@impala.apache.org. >> >> >> Thanks, >> >> Joe McDonnell >> >>
