[ https://issues.apache.org/jira/browse/IMPALA-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang updated IMPALA-9690: ----------------------------------- Labels: incompatibility performance (was: incompa performance) > Bump minimum x86-64 CPU requirements > ------------------------------------ > > Key: IMPALA-9690 > URL: https://issues.apache.org/jira/browse/IMPALA-9690 > Project: IMPALA > Issue Type: Improvement > Components: Backend > Reporter: Tim Armstrong > Priority: Blocker > Labels: incompatibility, performance > > We still have a minimum CPU requirement of SSSE3 support > https://impala.apache.org/docs/build/html/topics/impala_prereqs.html. I.e. we > don't assume SSE4.2 or AVX or AVX2. > There is a lot of legacy code to support CPUs without SSE4.2 and various > other extensions. As a start, here are all the locations in the code where we > branch based on CPU feature: > {noformat} > :~/impala/impala$ git grep CpuInfo::IsSupport > be/src/benchmarks/int-hash-benchmark.cc: if > (CpuInfo::IsSupported(CpuInfo::SSE4_2)) suite32.BENCH(uint32_t, CRC); > be/src/benchmarks/int-hash-benchmark.cc: if > (CpuInfo::IsSupported(CpuInfo::SSE4_2)) { > be/src/benchmarks/int-hash-benchmark.cc: if > (CpuInfo::IsSupported(CpuInfo::SSE4_1)) { > be/src/benchmarks/int-hash-benchmark.cc: if > (CpuInfo::IsSupported(CpuInfo::AVX2)) { > be/src/benchmarks/string-compare-benchmark.cc: if > (CpuInfo::IsSupported(CpuInfo::SSE4_2)) { > be/src/exec/delimited-text-parser.cc: if > (CpuInfo::IsSupported(CpuInfo::SSE4_2)) { > be/src/exec/delimited-text-parser.cc: if > (CpuInfo::IsSupported(CpuInfo::SSE4_2)) { > be/src/exec/delimited-text-parser.inline.h: > DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/exec/delimited-text-parser.inline.h: if > (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) { > be/src/runtime/io/disk-io-mgr.cc: if > (!CpuInfo::IsSupported(CpuInfo::SSE4_2)) { > be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::SSSE3)) { > be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) { > be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(cpu_info_flag)) { > be/src/util/bit-util-test.cc:// CpuInfo::IsSupported() checks. This doesn't > test the bug precisely but is a canary for > be/src/util/bit-util.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) { > be/src/util/bit-util.cc: } else if > (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) { > be/src/util/bit-util.cc: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) > { > be/src/util/bit-util.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::POPCNT))) > { > be/src/util/bloom-filter.cc: if (CpuInfo::IsSupported(CpuInfo::AVX)) { > be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) { > be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) { > be/src/util/cpu-info.cc: if (!CpuInfo::IsSupported(CpuInfo::SSSE3)) { > be/src/util/cpu-info.h: /// // line, CpuInfo::IsSupported(CpuInfo::AVX2) > will return false. > be/src/util/cpu-info.h: : feature_(feature), > reenable_(CpuInfo::IsSupported(feature)) { > be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2)); > be/src/util/hash-util.h: if > (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) { > be/src/util/openssl-util.cc: return > (CpuInfo::IsSupported(CpuInfo::PCLMULQDQ) > {noformat} > We also ship two versions of the codegen module, one of which (nosse42) is > essentially never used. > I think it would be uncontroversial to bump the minimum requirement to > SSE4.2, which would allow us to delete some old fallbacks. I think the last > time Intel or AMD shipped a processor without this was 2010 or 2011. Jumping > to AVX is probably almost as uncontroversial, since it looks like that has > been universal for nearly as long: > https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX. > Some older lower-performance cloud instance types don't support AVX, it looks > like, but I think this is an edge case. > It would be very nice to require AVX2 because that could remove a bunch of > conditional code (I think the contributors adding ARM support might want to > keep the scalar fallback code though, potentially). It looks like most Intel > and AMD processors have supported it since 2013 and 2015 respectively, except > some low-end Intel processors: > https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2. > In the past we've been a bit conservative about this - people do sometimes > use very old hardware as a test cluster because it was the hardware that they > could get their hands on (e.g. I remember someone complaining several years > back that they couldn't get Impala running on some old AMD CPUs that were > missing SSSE3 support). -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org