The UDA uses a STRING to store the intermediate value. The default if intermediate is not specified is for the intermediate type to be the same as the output type.
On Mon, Jun 24, 2019 at 11:31 PM skyyws <sky...@163.com> wrote: > Thanks for your advice, and this method works. > But I don’t understand why 'intermediate' is necessary for this uda, and > when do I need to add ‘intermediate’ in my own uda? > And I think this 'intermediate' is unfriendly to use, is this a history > problem? > > > | | > skyyws > | > | > sky...@163.com > | > 签名由网易邮箱大师定制 > > > On 06/25/2019 00:56，Tim Armstrong<tarmstr...@cloudera.com> wrote： > I think you need to declare the intermediate to be a string. > > E.g. here's a declaration of an aggregate function from our tets. > > create aggregate function agg_string_intermediate(decimal(20,10), bigint, > string) > returns decimal(20,0) intermediate string location '/path/to/lib.so' > init_fn='AggStringIntermediateInit' update_fn='AggStringIntermediateUpdate' > merge_fn='AggStringIntermediateMerge' > finalize_fn='AggStringIntermediateFinalize'; > > On Sun, Jun 23, 2019 at 3:36 AM skyyws <sky...@163.com> wrote: > > Hi all, > Recently, I tested the uda avg() in uda-sample.cc, and then impalad > crash. Here is the funciton created sql: > CREATE AGGREGATE FUNCTION avg_test(double) > RETURNS double > LOCATION '/user/impala/udf/libudasample.so' > UPDATE_FN='_Z9AvgUpdatePN10impala_udf15FunctionContextERKNS_9DoubleValEPPh' > INIT_FN='_Z7AvgInitPN10impala_udf15FunctionContextEPPh' > MERGE_FN='_Z8AvgMergePN10impala_udf15FunctionContextERKPhPS2_' > FINALIZE_FN='_Z11AvgFinalizePN10impala_udf15FunctionContextERKPh'; > and here is the error info in hs_err_pid*.log: > Stack: [0x00007f48b1c3a000,0x00007f48b243b000], sp=0x00007f48b24390e8, > free space=8188k > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > code) > C [libc.so.6+0x8570e] memset+0xee > C [impalad+0x1081e65] impala::AggFnEvaluator::Init(impala::Tuple*)+0x65 > C [impalad+0x1073efc] > impala::Aggregator::InitAggSlots(std::vector<impala::AggFnEvaluator*, > std::allocator<impala::AggFnEvaluator*> > const&, impala::Tuple*)+0x9c > C [impalad+0x106881d] > > impala::NonGroupingAggregator::ConstructSingletonOutputTuple(std::vector<impala::AggFnEvaluator*, > std::allocator<impala::AggFnEvaluator*> > const&, impala::MemPool*)+0xad > C [impalad+0x1068fa7] > impala::NonGroupingAggregator::Open(impala::RuntimeState*)+0x47 > C [impalad+0x1030f50] > impala::AggregationNode::Open(impala::RuntimeState*)+0x230 > C [impalad+0xb5e05e] impala::FragmentInstanceState::Open()+0x2ae > C [impalad+0xb5fc2d] impala::FragmentInstanceState::Exec()+0x1cd > C [impalad+0xb48002] > impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)+0x272 > C [impalad+0xd0a912] impala::Thread::SuperviseThread(std::string const&, > std::string const&, boost::function<void ()()>, impala::ThreadDebugInfo > const*, impala::Promise<long, (impala::PromiseMode)0>*)+0x2f2 > C [impalad+0xd0b45a] boost::detail::thread_data<boost::_bi::bind_t<void, > void (*)(std::string const&, std::string const&, boost::function<void > ()()>, impala::ThreadDebugInfo const*, impala::Promise<long, > (impala::PromiseMode)0>*), > boost::_bi::list5<boost::_bi::value<std::string>, > boost::_bi::value<std::string>, boost::_bi::value<boost::function<void > ()()> >, boost::_bi::value<impala::ThreadDebugInfo*>, > boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > > ::run()+0x7a > > but, when I tested count() in uda-sample.cc, everything is ok, and here is > the count() uda created sql: > > CREATE AGGREGATE FUNCTION count_test(bigint) > RETURNS bigint > LOCATION '/user/impala/udf/libudasample.so' > > > UPDATE_FN='_Z11CountUpdatePN10impala_udf15FunctionContextERKNS_6IntValEPNS_9BigIntValE' > INIT_FN='_Z9CountInitPN10impala_udf15FunctionContextEPNS_9BigIntValE' > > > MERGE_FN='_Z10CountMergePN10impala_udf15FunctionContextERKNS_9BigIntValEPS2_' > > > FINALIZE_FN='_Z13CountFinalizePN10impala_udf15FunctionContextERKNS_9BigIntValE'; > > > This problem happened both on version 3.1.0 and 2.12.0, and here is the > JIRA: https://issues.apache.org/jira/browse/IMPALA-8694 > > I’m not sure is this a bug or design for this? > | | > skyyws > | > | > sky...@163.com > | > > >
