[
https://issues.apache.org/jira/browse/IMPALA-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Hevele reassigned IMPALA-2744:
-------------------------------------
Assignee: Balazs Hevele
> Codegen tuple copy for Hash Join , Agg and possibly Exchange
> ------------------------------------------------------------
>
> Key: IMPALA-2744
> URL: https://issues.apache.org/jira/browse/IMPALA-2744
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.2
> Reporter: Mostafa Mokhtar
> Assignee: Balazs Hevele
> Priority: Major
> Labels: codegen, performance
>
> Join, Agg and exchange operators make tuple copies which are expensive.
> Consider codegen the copy to mitigate the cost
> {code}
> Data Of Interest (CPU Metrics)
> 1 of 98: 28.9% (5.010s of 17.338s)
> libc.so.6!memcpy - [Unknown]
> impalad!impala::BufferedTupleStream::DeepCopyInternal<(bool)0>+0xa2 -
> [Unknown]:[Unknown]
> libc.so.6![Unknown stack frame(s)] - [Unknown]:[Unknown]
> impalad!impala::PartitionedHashJoinNode::ConstructBuildSide+0x106 -
> [Unknown]:[Unknown]
> impalad!impala::BlockingJoinNode::BuildSideThread+0x7f - [Unknown]:[Unknown]
> impalad!impala::Thread::SuperviseThread+0x1b9 - [Unknown]:[Unknown]
> impalad!boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void (void)>,
> impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void
> (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run+0x7f -
> [Unknown]:[Unknown]
> impalad!func@0xd27210+0x62 - [Unknown]:[Unknown]
> libpthread.so.0!start_thread+0xd0 - [Unknown]:[Unknown]
> libc.so.6!clone+0x6c - [Unknown]:[Unknown]
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 2 of 98: 8.4% (1.459s of 17.338s)
> libc.so.6!memcpy - [Unknown]
> impalad!impala::BufferedTupleStream::CopyStrings+0x9b - [Unknown]:[Unknown]
> impalad!impala::BufferedTupleStream::DeepCopyInternal<(bool)0>+0xed -
> [Unknown]:[Unknown]
> libc.so.6![Unknown stack frame(s)] - [Unknown]:[Unknown]
> impalad!impala::PartitionedHashJoinNode::ConstructBuildSide+0x106 -
> [Unknown]:[Unknown]
> impalad!impala::BlockingJoinNode::BuildSideThread+0x7f - [Unknown]:[Unknown]
> impalad!impala::Thread::SuperviseThread+0x1b9 - [Unknown]:[Unknown]
> impalad!boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void (void)>,
> impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void
> (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run+0x7f -
> [Unknown]:[Unknown]
> impalad!func@0xd27210+0x62 - [Unknown]:[Unknown]
> libpthread.so.0!start_thread+0xd0 - [Unknown]:[Unknown]
> libc.so.6!clone+0x6c - [Unknown]:[Unknown]
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 3 of 98: 6.9% (1.190s of 17.338s)
> libc.so.6!memcpy - [Unknown]
> impalad!impala::BufferedTupleStream::DeepCopyInternal<(bool)0>+0xa2 -
> [Unknown]:[Unknown]
> impalad!impala::BufferedTupleStream::DeepCopyInternal<(bool)0>+0xa2 -
> [Unknown]:[Unknown]
> libc.so.6![Unknown stack frame(s)] - [Unknown]:[Unknown]
> impalad!impala::PartitionedHashJoinNode::ConstructBuildSide+0x106 -
> [Unknown]:[Unknown]
> impalad!impala::BlockingJoinNode::BuildSideThread+0x7f - [Unknown]:[Unknown]
> impalad!impala::Thread::SuperviseThread+0x1b9 - [Unknown]:[Unknown]
> impalad!boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void (void)>,
> impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void
> (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run+0x7f -
> [Unknown]:[Unknown]
> impalad!func@0xd27210+0x62 - [Unknown]:[Unknown]
> libpthread.so.0!start_thread+0xd0 - [Unknown]:[Unknown]
> libc.so.6!clone+0x6c - [Unknown]:[Unknown]
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 4 of 98: 6.7% (1.169s of 17.338s)
> libc.so.6!memcpy - [Unknown]
> impalad!impala::ExchangeNode::GetNext+0x5bf - [Unknown]:[Unknown]
> impalad!impala::PartitionedHashJoinNode::ProcessBuildInput+0x4b7 -
> [Unknown]:[Unknown]
> impalad!impala::PartitionedHashJoinNode::ConstructBuildSide+0x106 -
> [Unknown]:[Unknown]
> impalad!impala::BlockingJoinNode::BuildSideThread+0x7f - [Unknown]:[Unknown]
> impalad!impala::Thread::SuperviseThread+0x1b9 - [Unknown]:[Unknown]
> impalad!boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void (void)>,
> impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void
> (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run+0x7f -
> [Unknown]:[Unknown]
> impalad!func@0xd27210+0x62 - [Unknown]:[Unknown]
> libpthread.so.0!start_thread+0xd0 - [Unknown]:[Unknown]
> libc.so.6!clone+0x6c - [Unknown]:[Unknown]
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 5 of 98: 6.3% (1.100s of 17.338s)
> libc.so.6!memcpy - [Unknown]
> impalad!apache::thrift::transport::TVirtualTransport<apache::thrift::transport::TBufferedTransport,
> apache::thrift::transport::TBufferBase>::write_virt+0x27 -
> [Unknown]:[Unknown]
> impalad!apache::thrift::protocol::TVirtualProtocol<apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport>,
> apache::thrift::protocol::TProtocolDefaults>::writeI32_virt+0x1d -
> [Unknown]:[Unknown]
> impalad!impala::TRowBatch::write+0x14b - [Unknown]:[Unknown]
> impalad!impala::TTransmitDataParams::write+0x128 - [Unknown]:[Unknown]
> impalad!impala::ImpalaInternalService_TransmitData_pargs::write+0x59 -
> [Unknown]:[Unknown]
> impalad!impala::ImpalaInternalServiceClient::send_TransmitData+0x65 -
> [Unknown]:[Unknown]
> impalad!impala::ImpalaInternalServiceClient::TransmitData+0x1b -
> [Unknown]:[Unknown]
> impalad!impala::ClientConnection<impala::ImpalaInternalServiceClient>::DoRpc<void
> (impala::TTransmitDataResult&, impala::TTransmitDataParams const&)
> impala::ImpalaInternalServiceClient::*, impala::TTransmitDataParams,
> impala::TTransmitDataResult>+0x4a - [Unknown]:[Unknown]
> impalad!impala::DataStreamSender::Channel::TransmitDataHelper+0x5b1 -
> [Unknown]:[Unknown]
> impalad!impala::DataStreamSender::Channel::TransmitData+0x10 -
> [Unknown]:[Unknown]
> impalad!impala::ThreadPool<impala::TRowBatch*>::WorkerThread+0x2f4 -
> [Unknown]:[Unknown]
> impalad!impala::Thread::SuperviseThread+0x1b9 - [Unknown]:[Unknown]
> impalad!boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void (void)>,
> impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void
> (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run+0x7f -
> [Unknown]:[Unknown]
> impalad!func@0xd27210+0x62 - [Unknown]:[Unknown]
> libpthread.so.0!start_thread+0xd0 - [Unknown]:[Unknown]
> libc.so.6!clone+0x6c - [Unknown]:[Unknown]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
