[ https://issues.apache.org/jira/browse/IMPALA-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gabor Kaszab resolved IMPALA-2603. ---------------------------------- Resolution: Fixed > Incorrect results and plan for inline view referencing several collection > types correlated with different ancestor blocks > ------------------------------------------------------------------------------------------------------------------------- > > Key: IMPALA-2603 > URL: https://issues.apache.org/jira/browse/IMPALA-2603 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Affects Versions: Impala 2.3.0 > Reporter: Taras Bobrovytsky > Priority: Critical > Labels: complextype, correctness, crash, downgraded, > nested_types, query_generator, ramp-up > > *Problem* > Queries with multiple nested inline views that have correlated references to > nested collections (relative table references), can return incorrect results > in RELEASE and hit a DCHECK in debug under the following condition: > * There is an inline view that references multiple nested collections which > come from different ancestor blocks at different levels of nesting. > * In the example below, Impala fails to generate a correct plan for the "a" > inline view because the references "t5" and "t6" reference different ancestor > query blocks at different nesting levels. > Query: > {code} > SELECT > 1 > FROM > customer t1 > INNER JOIN ( > SELECT > 1 > FROM > t1.c_orders t2 > INNER JOIN ( > SELECT > 1 > FROM > t2.o_lineitems t5 > INNER JOIN t1.c_orders t6 > ) as a > ) as b; > {code} > Wrong Query Plan: > {code} > +------------------------------------------------------------------------------------+ > | Explain String > | > +------------------------------------------------------------------------------------+ > | Estimated Per-Host Requirements: Memory=176.00MB VCores=1 > | > | WARNING: The following tables are missing relevant table and/or column > statistics. | > | tpch_nested_parquet.customer > | > | > | > | 05:EXCHANGE [UNPARTITIONED] > | > | | > | > | 01:SUBPLAN > | > | | > | > | |--04:NESTED LOOP JOIN [CROSS JOIN] > | > | | | > | > | | |--02:SINGULAR ROW SRC > | > | | | > | > | | 03:UNNEST [t1.c_orders t2] > | > | | > | > | 00:SCAN HDFS [tpch_nested_parquet.customer t1] > | > | partitions=1/1 files=4 size=554.13MB > | > +------------------------------------------------------------------------------------+ > {code} > Stack Trace: > {code} > #0 0x00007f5c10cf5cc9 in __GI_raise (sig=sig@entry=6) at > ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x00007f5c10cf90d8 in __GI_abort () at abort.c:89 > #2 0x0000000002144d09 in google::DumpStackTraceAndExit () at > src/utilities.cc:147 > #3 0x000000000213ddbd in google::LogMessage::Fail () at src/logging.cc:1315 > #4 0x000000000213fc45 in google::LogMessage::SendToLog (this=0x7f5b9ef08e00) > at src/logging.cc:1269 > #5 0x000000000213d913 in google::LogMessage::Flush > (this=this@entry=0x7f5b9ef08e00) at src/logging.cc:1138 > #6 0x000000000214059e in google::LogMessageFatal::~LogMessageFatal > (this=0x7f5b9ef08e00, __in_chrg=<optimized out>) at src/logging.cc:1836 > #7 0x0000000001586657 in impala::Coordinator::ValidateCollectionSlots > (this=0xc49ca00, batch=0xc3b3e00) at > /home/dev/Impala/be/src/runtime/coordinator.cc:911 > #8 0x000000000158638d in impala::Coordinator::GetNext (this=0xc49ca00, > batch=0x7dd3bd0, state=0xd934400) at > /home/dev/Impala/be/src/runtime/coordinator.cc:890 > #9 0x00000000013710c3 in > impala::ImpalaServer::QueryExecState::FetchNextBatch (this=0x7dd2000) at > /home/dev/Impala/be/src/service/query-exec-state.cc:877 > #10 0x000000000136f169 in > impala::ImpalaServer::QueryExecState::FetchRowsInternal (this=0x7dd2000, > max_rows=1024, fetched_rows=0x7f5b9ef09370) at > /home/dev/Impala/be/src/service/query-exec-state.cc:706 > #11 0x000000000136e80d in impala::ImpalaServer::QueryExecState::FetchRows > (this=0x7dd2000, max_rows=1024, fetched_rows=0x7f5b9ef09370) at > /home/dev/Impala/be/src/service/query-exec-state.cc:616 > #12 0x0000000001364418 in impala::ImpalaServer::FetchInternal > (this=0x910ac00, query_id=..., start_over=false, fetch_size=1024, > query_results=0x7f5b9ef09668) at > /home/dev/Impala/be/src/service/impala-beeswax-server.cc:613 > #13 0x000000000135fd34 in impala::ImpalaServer::fetch (this=0x910ac00, > query_results=..., query_handle=..., start_over=false, fetch_size=1024) at > /home/dev/Impala/be/src/service/impala-beeswax-server.cc:288 > #14 0x00000000014fded4 in beeswax::BeeswaxServiceProcessor::process_fetch > (this=0xc9fdf40, seqid=0, iprot=0x7d789c0, oprot=0x8fe8340, > callContext=0x7efe9c0) at > /home/dev/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:3150 > #15 0x00000000014fc312 in beeswax::BeeswaxServiceProcessor::dispatchCall > (this=0xc9fdf40, iprot=0x7d789c0, oprot=0x8fe8340, fname=..., seqid=0, > callContext=0x7efe9c0) at > /home/dev/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:2952 > #16 0x00000000014e577b in impala::ImpalaServiceProcessor::dispatchCall > (this=0xc9fdf40, iprot=0x7d789c0, oprot=0x8fe8340, fname=..., seqid=0, > callContext=0x7efe9c0) at > /home/dev/Impala/be/generated-sources/gen-cpp/ImpalaService.cpp:1673 > #17 0x00000000012fac15 in apache::thrift::TDispatchProcessor::process > (this=0xc9fdf40, in=..., out=..., connectionContext=0x7efe9c0) at > /home/dev/Impala/thirdparty/thrift-0.9.0/build/include/thrift/TDispatchProcessor.h:121 > #18 0x00000000020e39ea in > apache::thrift::server::TThreadPoolServer::Task::run (this=0xcd9bbc0) at > src/thrift/server/TThreadPoolServer.cpp:70 > #19 0x00000000020d012f in > apache::thrift::concurrency::ThreadManager::Task::run (this=0x8fe8300) at > src/thrift/concurrency/ThreadManager.cpp:187 > #20 0x00000000020d2d9a in > apache::thrift::concurrency::ThreadManager::Worker::run (this=0xce50f90) at > src/thrift/concurrency/ThreadManager.cpp:316 > #21 0x00000000012241a5 in impala::ThriftThread::RunRunnable (this=0x90d5a80, > runnable=..., promise=0x7fffce7e4120) at > /home/dev/Impala/be/src/rpc/thrift-thread.cc:61 > #22 0x000000000122596d in boost::_mfi::mf2<void, impala::ThriftThread, > boost::shared_ptr<apache::thrift::concurrency::Runnable>, > impala::Promise<unsigned long>*>::operator() (this=0x7f00780, p=0x90d5a80, > a1=..., a2=0x7fffce7e4120) at /usr/include/boost/bind/mem_fn_template.hpp:280 > #23 0x00000000012257c8 in > boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, > boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, > boost::_bi::value<impala::Promise<unsigned long>*> > >::operator()<boost::_mfi::mf2<void, impala::ThriftThread, > boost::shared_ptr<apache::thrift::concurrency::Runnable>, > impala::Promise<unsigned long>*>, boost::_bi::list0> (this=0x7f00790, f=..., > a=...) at /usr/include/boost/bind/bind.hpp:392 > #24 0x0000000001225559 in boost::_bi::bind_t<void, boost::_mfi::mf2<void, > impala::ThriftThread, > boost::shared_ptr<apache::thrift::concurrency::Runnable>, > impala::Promise<unsigned long>*>, > boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, > boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, > boost::_bi::value<impala::Promise<unsigned long>*> > >::operator() > (this=0x7f00780) at /usr/include/boost/bind/bind_template.hpp:20 > #25 0x000000000122547a in > boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, > boost::_mfi::mf2<void, impala::ThriftThread, > boost::shared_ptr<apache::thrift::concurrency::Runnable>, > impala::Promise<unsigned long>*>, > boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, > boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, > boost::_bi::value<impala::Promise<unsigned long>*> > >, void>::invoke > (function_obj_ptr=...) at > /usr/include/boost/function/function_template.hpp:153 > #26 0x000000000125b4a8 in boost::function0<void>::operator() > (this=0x7f5b9ef09e00) at /usr/include/boost/function/function_template.hpp:767 > #27 0x000000000146e091 in impala::Thread::SuperviseThread(std::string const&, > std::string const&, boost::function<void ()>, impala::Promise<long>*) > (name=..., category=..., functor=..., thread_started=0x7fffce7e3f60) at > /home/dev/Impala/be/src/util/thread.cc:314 > #28 0x00000000014772c7 in boost::_bi::list4<boost::_bi::value<std::string>, > boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, > boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string > const&, std::string const&, boost::function<void ()>, > impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void > (*&)(std::string const&, std::string const&, boost::function<void ()>, > impala::Promise<long>*), boost::_bi::list0&, int) (this=0x90d85c0, > f=@0x90d85b8: 0x146dd88 <impala::Thread::SuperviseThread(std::string const&, > std::string const&, boost::function<void ()>, impala::Promise<long>*)>, > a=...) at /usr/include/boost/bind/bind.hpp:457 > #29 0x0000000001477211 in boost::_bi::bind_t<void, void (*)(std::string > const&, std::string const&, boost::function<void ()>, > impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, > boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, > boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0x90d85b8) > at /usr/include/boost/bind/bind_template.hpp:20 > #30 0x00000000014771d6 in boost::detail::thread_data<boost::_bi::bind_t<void, > void (*)(std::string const&, std::string const&, boost::function<void ()>, > impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, > boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, > boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0x90d8400) at > /usr/include/boost/thread/detail/thread.hpp:117 > #31 0x00007f5c13cbfa4a in ?? () from > /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0 > #32 0x00007f5c1316c182 in start_thread (arg=0x7f5b9ef0a700) at > pthread_create.c:312 > #33 0x00007f5c10db947d in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 > {code} > impalad.FATAL: > {code} > Check failed: tuple->IsNull(slot_desc->null_indicator_offset()) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org