[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5356: Hadoop Flags: Incompatible change > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.10.patch, > HIVE-5356.11.patch, HIVE-5356.12.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to trunk. Thank Brock for the review. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.10.patch, > HIVE-5356.11.patch, HIVE-5356.12.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.12.patch Patch #12 addressed the new test failures and provided a temp fix for it. The correct fix will be provided in HIVE-5848. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.10.patch, > HIVE-5356.11.patch, HIVE-5356.12.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.11.patch Patch #11 rebased with latest trunk. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.10.patch, > HIVE-5356.11.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, HIVE-5356.4.patch, > HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, HIVE-5356.8.patch, > HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.10.patch Patch #10 fixed a bug and added more test cases. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.10.patch, > HIVE-5356.2.patch, HIVE-5356.3.patch, HIVE-5356.4.patch, HIVE-5356.5.patch, > HIVE-5356.6.patch, HIVE-5356.7.patch, HIVE-5356.8.patch, HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: (was: HIVE-5356.patch) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.9.patch > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: (was: HIVE-5356.9.patch) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.9.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.9.patch Patch #9 removed a few trailing spaces. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.9.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.8.patch Patch #8, Minor format cleanup. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.8.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.7.patch Patch #7 attempted to fix the test failures. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.7.patch, > HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.6.patch Patch #6 is equivalent to #5 with some minor refactoring. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.6.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.5.patch > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: (was: HIVE-5356.5.patch) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.5.patch Patch #5 fixes remaining test failures. Still not final yet. Some refactoring pending. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.5.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.4.patch Patch #5 rebased with latest trunk (again). > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.4.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Status: Patch Available (was: Open) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.3.patch Patch # is updated with fixes for a bunch of test failures. Submitted patch to trigger another test run. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.3.patch, > HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Status: Open (was: Patch Available) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.2.patch Patch #3 included additional tests, fixes. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Status: Patch Available (was: Open) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.1.patch Patch #2 is rebased and updated. Still missing test cases. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.1.patch, HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Status: Open (was: Patch Available) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Attachment: HIVE-5356.patch > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Status: Patch Available (was: Open) > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > Attachments: HIVE-5356.patch > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
[ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5356: -- Description: Currently, all of the arithmetic operators, such as add/sub/mult/div, are implemented as old-style UDFs and java reflection is used to determine the return type TypeInfos/ObjectInspectors, based on the return type of the evaluate() method chosen for the expression. This works fine for types that don't have type params. Hive decimal type participates in these operations just like int or double. Different from double or int, however, decimal has precision and scale, which cannot be determined by just looking at the return type (decimal) of the UDF evaluate() method, even though the operands have certain precision/scale. With the default of "decimal" without precision/scale, then (10, 0) will be the type params. This is certainly not desirable. To solve this problem, all of the arithmetic operators would need to be implemented as GenericUDFs, which allow returning ObjectInspector during the initialize() method. The object inspectors returned can carry type params, from which the "exact" return type can be determined. It's worth mentioning that, for user UDF implemented in non-generic way, if the return type of the chosen evaluate() method is decimal, the return type actually has (10,0) as precision/scale, which might not be desirable. This needs to be documented. This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit the scope of review. The remaining ones will be covered under HIVE-5706. was: Currently, all of the arithmetic operators, such as add/sub/mult/div, are implemented as old-style UDFs and java reflection is used to determine the return type TypeInfos/ObjectInspectors, based on the return type of the evaluate() method chosen for the expression. This works fine for types that don't have type params. Hive decimal type participates in these operations just like int or double. Different from double or int, however, decimal has precision and scale, which cannot be determined by just looking at the return type (decimal) of the UDF evaluate() method, even though the operands have certain precision/scale. With the default of "decimal" without precision/scale, then (10, 0) will be the type params. This is certainly not desirable. To solve this problem, all of the arithmetic operators would need to be implemented as GenericUDFs, which allow returning ObjectInspector during the initialize() method. The object inspectors returned can carry type params, from which the "exact" return type can be determined. It's worth mentioning that, for user UDF implemented in non-generic way, if the return type of the chosen evaluate() method is decimal, the return type actually has (10,0) as precision/scale, which might not be desirable. This needs to be documented. > Move arithmatic UDFs to generic UDF implementations > --- > > Key: HIVE-5356 > URL: https://issues.apache.org/jira/browse/HIVE-5356 > Project: Hive > Issue Type: Task > Components: UDF >Affects Versions: 0.11.0 >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > Fix For: 0.13.0 > > > Currently, all of the arithmetic operators, such as add/sub/mult/div, are > implemented as old-style UDFs and java reflection is used to determine the > return type TypeInfos/ObjectInspectors, based on the return type of the > evaluate() method chosen for the expression. This works fine for types that > don't have type params. > Hive decimal type participates in these operations just like int or double. > Different from double or int, however, decimal has precision and scale, which > cannot be determined by just looking at the return type (decimal) of the UDF > evaluate() method, even though the operands have certain precision/scale. > With the default of "decimal" without precision/scale, then (10, 0) will be > the type params. This is certainly not desirable. > To solve this problem, all of the arithmetic operators would need to be > implemented as GenericUDFs, which allow returning ObjectInspector during the > initialize() method. The object inspectors returned can carry type params, > from which the "exact" return type can be determined. > It's worth mentioning that, for user UDF implemented in non-generic way, if > the return type of the chosen evaluate() method is decimal, the return type > actually has (10,0) as precision/scale, which might not be desirable. This > needs to be documented. > This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit > the scope of review. The remaining ones will be covered under HIVE-5706. -- This message was sent by Atlassian JIRA (v6.1#6144)
