[
https://issues.apache.org/jira/browse/HIVE-27960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gaoxiong updated HIVE-27960:
----------------------------
Description:
When a permanent udaf used before over() function, hive will throw invalid
function error.
-In HIVE-12719 , it fix this issue for hive 3, but it can't work in hive 2.-
This issue reproduce on master.
In hive 2, it should get FunctionInfo from FunctionRegistry before get
WindowFunctionInfo same to hive 3. Because it will register window function to
session. Then hive can get WindowFunctionInfo correctly.
---------------------------- err detail ----------------------------
register a permanent udaf:
{code:java}
create function row_number2 as
'org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRowNumber'; {code}
execute query in a new cli session:
{code:java}
select row_number2() over();{code}
blew is error log:
{code:java}
FAILED: SemanticException Failed to breakup Windowing invocations into Groups.
At least 1 group must only depend on input columns. Also check for circular
dependencies.Underlying error: Invalid function
row_number22023-12-06T10:17:30,348 ERROR [0b7764ce-cde3-49c5-9d32-f96d61b20773
main] ql.Driver: FAILED: SemanticException Failed to breakup Windowing
invocations into Groups. At least 1 group must only depend on input columns.
Also check for circular dependencies.Underlying error: Invalid function
row_number2org.apache.hadoop.hive.ql.parse.SemanticException: Failed to breakup
Windowing invocations into Groups. At least 1 group must only depend on input
columns. Also check for circular dependencies.Underlying error: Invalid
function row_number2 at
org.apache.hadoop.hive.ql.parse.WindowingComponentizer.next(WindowingComponentizer.java:97)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genWindowingPlan(SemanticAnalyzer.java:13270)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:9685)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:9644)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:10549)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:10427)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:11125)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:481)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) at
org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.RunJar.run(RunJar.java:239) at
org.apache.hadoop.util.RunJar.main(RunJar.java:153) {code}
was:
When a permanent udaf used before over() function, hive will throw invalid
function error.
In HIVE-12719 , it fix this issue for hive 3, but it can't work in hive 2.
In hive 2, it should get FunctionInfo from FunctionRegistry before get
WindowFunctionInfo same to hive 3. Because it will register window function to
session. Then hive can get WindowFunctionInfo correctly.
---------------------------- err detail ----------------------------
register a permanent udaf:
{code:java}
create function row_number2 as
'org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRowNumber'; {code}
execute query in a new cli session:
{code:java}
select row_number2() over();{code}
blew is error log:
{code:java}
FAILED: SemanticException Failed to breakup Windowing invocations into Groups.
At least 1 group must only depend on input columns. Also check for circular
dependencies.Underlying error: Invalid function
row_number22023-12-06T10:17:30,348 ERROR [0b7764ce-cde3-49c5-9d32-f96d61b20773
main] ql.Driver: FAILED: SemanticException Failed to breakup Windowing
invocations into Groups. At least 1 group must only depend on input columns.
Also check for circular dependencies.Underlying error: Invalid function
row_number2org.apache.hadoop.hive.ql.parse.SemanticException: Failed to breakup
Windowing invocations into Groups. At least 1 group must only depend on input
columns. Also check for circular dependencies.Underlying error: Invalid
function row_number2 at
org.apache.hadoop.hive.ql.parse.WindowingComponentizer.next(WindowingComponentizer.java:97)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genWindowingPlan(SemanticAnalyzer.java:13270)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:9685)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:9644)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:10549)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:10427)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:11125)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:481)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) at
org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.RunJar.run(RunJar.java:239) at
org.apache.hadoop.util.RunJar.main(RunJar.java:153) {code}
> Invalid function error when using custom udaf
> ---------------------------------------------
>
> Key: HIVE-27960
> URL: https://issues.apache.org/jira/browse/HIVE-27960
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 2.3.9
> Environment: Aliyun emr hive 2.3.9
> Reporter: gaoxiong
> Assignee: gaoxiong
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.9
>
>
> When a permanent udaf used before over() function, hive will throw invalid
> function error.
>
> -In HIVE-12719 , it fix this issue for hive 3, but it can't work in hive 2.-
> This issue reproduce on master.
>
> In hive 2, it should get FunctionInfo from FunctionRegistry before get
> WindowFunctionInfo same to hive 3. Because it will register window function
> to session. Then hive can get WindowFunctionInfo correctly.
>
> ---------------------------- err detail ----------------------------
> register a permanent udaf:
> {code:java}
> create function row_number2 as
> 'org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRowNumber'; {code}
> execute query in a new cli session:
> {code:java}
> select row_number2() over();{code}
> blew is error log:
> {code:java}
> FAILED: SemanticException Failed to breakup Windowing invocations into
> Groups. At least 1 group must only depend on input columns. Also check for
> circular dependencies.Underlying error: Invalid function
> row_number22023-12-06T10:17:30,348 ERROR
> [0b7764ce-cde3-49c5-9d32-f96d61b20773 main] ql.Driver: FAILED:
> SemanticException Failed to breakup Windowing invocations into Groups. At
> least 1 group must only depend on input columns. Also check for circular
> dependencies.Underlying error: Invalid function
> row_number2org.apache.hadoop.hive.ql.parse.SemanticException: Failed to
> breakup Windowing invocations into Groups. At least 1 group must only depend
> on input columns. Also check for circular dependencies.Underlying error:
> Invalid function row_number2 at
> org.apache.hadoop.hive.ql.parse.WindowingComponentizer.next(WindowingComponentizer.java:97)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genWindowingPlan(SemanticAnalyzer.java:13270)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:9685)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:9644)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:10549)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:10427)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:11125)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:481)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512) at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) at
> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at
> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.util.RunJar.run(RunJar.java:239) at
> org.apache.hadoop.util.RunJar.main(RunJar.java:153) {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
