[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-2249:
---
Fix Version/s: 0.9.0
When creating constant expression for numbers, try to infer type from another
comparison operand, instead of trying to use integer first, and then long and
double
--
Key: HIVE-2249
URL: https://issues.apache.org/jira/browse/HIVE-2249
Project: Hive
Issue Type: Improvement
Reporter: Siying Dong
Assignee: Zhiqiu Kong
Fix For: 0.9.0
Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt,
HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch,
HIVE-2249.D1383.4.patch
The current code to build constant expression for numbers, here is the code:
try {
v = Double.valueOf(expr.getText());
v = Long.valueOf(expr.getText());
v = Integer.valueOf(expr.getText());
} catch (NumberFormatException e) {
// do nothing here, we will throw an exception in the following block
}
if (v == null) {
throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
.getMsg(expr));
}
return new ExprNodeConstantDesc(v);
The for the case that WHERE BIG_INT_COLUMN = 0, or WHERE DOUBLE_COLUMN
= 0, we always have to do a type conversion when comparing, which is
unnecessary if it is slightly smarter to choose type when creating the
constant expression. We can simply walk one level up the tree, find another
comparison party and use the same type with that one if it is possible. For
user's wrong query like 'INT_COLUMN=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2249: -- Attachment: HIVE-2249.D1383.4.patch zhiqiu updated the revision HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double. Reviewers: njain, kevinwilfong, heyongqiang, JIRA, stuart983 Fix insert1_overwrite_partitions.q and insert2_overwrite_partitions.q thus the results are deterministic REVISION DETAIL https://reviews.facebook.net/D1383 AFFECTED FILES contrib/src/test/results/clientpositive/dboutput.q.out contrib/src/test/results/clientpositive/serde_typedbytes4.q.out data/files/infer_const_type.txt ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java ql/src/test/queries/clientpositive/infer_const_type.q ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q ql/src/test/queries/clientpositive/ppr_pushdown.q ql/src/test/results/clientpositive/auto_join0.q.out ql/src/test/results/clientpositive/auto_join11.q.out ql/src/test/results/clientpositive/auto_join12.q.out ql/src/test/results/clientpositive/auto_join13.q.out ql/src/test/results/clientpositive/auto_join14.q.out ql/src/test/results/clientpositive/auto_join16.q.out ql/src/test/results/clientpositive/auto_join20.q.out ql/src/test/results/clientpositive/auto_join21.q.out ql/src/test/results/clientpositive/auto_join23.q.out ql/src/test/results/clientpositive/auto_join27.q.out ql/src/test/results/clientpositive/auto_join28.q.out ql/src/test/results/clientpositive/auto_join29.q.out ql/src/test/results/clientpositive/auto_join4.q.out ql/src/test/results/clientpositive/auto_join5.q.out ql/src/test/results/clientpositive/auto_join6.q.out ql/src/test/results/clientpositive/auto_join7.q.out ql/src/test/results/clientpositive/auto_join8.q.out ql/src/test/results/clientpositive/cast1.q.out ql/src/test/results/clientpositive/cluster.q.out ql/src/test/results/clientpositive/create_view.q.out ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out ql/src/test/results/clientpositive/having.q.out ql/src/test/results/clientpositive/index_auto.q.out ql/src/test/results/clientpositive/index_auto_empty.q.out ql/src/test/results/clientpositive/index_auto_file_format.q.out ql/src/test/results/clientpositive/index_auto_mult_tables.q.out ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out ql/src/test/results/clientpositive/index_auto_multiple.q.out ql/src/test/results/clientpositive/index_auto_partitioned.q.out ql/src/test/results/clientpositive/index_auto_self_join.q.out ql/src/test/results/clientpositive/index_auto_unused.q.out ql/src/test/results/clientpositive/index_auto_update.q.out ql/src/test/results/clientpositive/index_bitmap3.q.out ql/src/test/results/clientpositive/index_bitmap_auto.q.out ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out ql/src/test/results/clientpositive/index_bitmap_compression.q.out ql/src/test/results/clientpositive/index_compression.q.out ql/src/test/results/clientpositive/index_stale.q.out ql/src/test/results/clientpositive/index_stale_partitioned.q.out ql/src/test/results/clientpositive/infer_const_type.q.out ql/src/test/results/clientpositive/input11.q.out ql/src/test/results/clientpositive/input11_limit.q.out ql/src/test/results/clientpositive/input12.q.out ql/src/test/results/clientpositive/input13.q.out ql/src/test/results/clientpositive/input14.q.out ql/src/test/results/clientpositive/input14_limit.q.out ql/src/test/results/clientpositive/input18.q.out ql/src/test/results/clientpositive/input1_limit.q.out ql/src/test/results/clientpositive/input2_limit.q.out ql/src/test/results/clientpositive/input42.q.out ql/src/test/results/clientpositive/input_part1.q.out ql/src/test/results/clientpositive/input_part2.q.out ql/src/test/results/clientpositive/input_part5.q.out ql/src/test/results/clientpositive/input_part7.q.out ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out ql/src/test/results/clientpositive/join0.q.out ql/src/test/results/clientpositive/join11.q.out ql/src/test/results/clientpositive/join12.q.out ql/src/test/results/clientpositive/join13.q.out ql/src/test/results/clientpositive/join14.q.out ql/src/test/results/clientpositive/join16.q.out ql/src/test/results/clientpositive/join20.q.out ql/src/test/results/clientpositive/join21.q.out ql/src/test/results/clientpositive/join23.q.out ql/src/test/results/clientpositive/join34.q.out ql/src/test/results/clientpositive/join35.q.out ql/src/test/results/clientpositive/join38.q.out
[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2249: -- Attachment: HIVE-2249.D1383.3.patch zhiqiu updated the revision HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double. Reviewers: njain, kevinwilfong, heyongqiang, JIRA, stuart983 Address Siying's and Kevin's comments * For string_col = long_const, no smart type inference is done * Add query to test case to verify the results are correct REVISION DETAIL https://reviews.facebook.net/D1383 AFFECTED FILES contrib/src/test/results/clientpositive/dboutput.q.out contrib/src/test/results/clientpositive/serde_typedbytes4.q.out data/files/infer_const_type.txt ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java ql/src/test/queries/clientpositive/infer_const_type.q ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q ql/src/test/queries/clientpositive/ppr_pushdown.q ql/src/test/results/clientpositive/auto_join0.q.out ql/src/test/results/clientpositive/auto_join11.q.out ql/src/test/results/clientpositive/auto_join12.q.out ql/src/test/results/clientpositive/auto_join13.q.out ql/src/test/results/clientpositive/auto_join14.q.out ql/src/test/results/clientpositive/auto_join16.q.out ql/src/test/results/clientpositive/auto_join20.q.out ql/src/test/results/clientpositive/auto_join21.q.out ql/src/test/results/clientpositive/auto_join23.q.out ql/src/test/results/clientpositive/auto_join27.q.out ql/src/test/results/clientpositive/auto_join28.q.out ql/src/test/results/clientpositive/auto_join29.q.out ql/src/test/results/clientpositive/auto_join4.q.out ql/src/test/results/clientpositive/auto_join5.q.out ql/src/test/results/clientpositive/auto_join6.q.out ql/src/test/results/clientpositive/auto_join7.q.out ql/src/test/results/clientpositive/auto_join8.q.out ql/src/test/results/clientpositive/cast1.q.out ql/src/test/results/clientpositive/cluster.q.out ql/src/test/results/clientpositive/create_view.q.out ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out ql/src/test/results/clientpositive/having.q.out ql/src/test/results/clientpositive/index_auto.q.out ql/src/test/results/clientpositive/index_auto_empty.q.out ql/src/test/results/clientpositive/index_auto_file_format.q.out ql/src/test/results/clientpositive/index_auto_mult_tables.q.out ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out ql/src/test/results/clientpositive/index_auto_multiple.q.out ql/src/test/results/clientpositive/index_auto_partitioned.q.out ql/src/test/results/clientpositive/index_auto_self_join.q.out ql/src/test/results/clientpositive/index_auto_unused.q.out ql/src/test/results/clientpositive/index_auto_update.q.out ql/src/test/results/clientpositive/index_bitmap3.q.out ql/src/test/results/clientpositive/index_bitmap_auto.q.out ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out ql/src/test/results/clientpositive/index_bitmap_compression.q.out ql/src/test/results/clientpositive/index_compression.q.out ql/src/test/results/clientpositive/index_stale.q.out ql/src/test/results/clientpositive/index_stale_partitioned.q.out ql/src/test/results/clientpositive/infer_const_type.q.out ql/src/test/results/clientpositive/input11.q.out ql/src/test/results/clientpositive/input11_limit.q.out ql/src/test/results/clientpositive/input12.q.out ql/src/test/results/clientpositive/input13.q.out ql/src/test/results/clientpositive/input14.q.out ql/src/test/results/clientpositive/input14_limit.q.out ql/src/test/results/clientpositive/input18.q.out ql/src/test/results/clientpositive/input1_limit.q.out ql/src/test/results/clientpositive/input2_limit.q.out ql/src/test/results/clientpositive/input42.q.out ql/src/test/results/clientpositive/input_part1.q.out ql/src/test/results/clientpositive/input_part2.q.out ql/src/test/results/clientpositive/input_part5.q.out ql/src/test/results/clientpositive/input_part7.q.out ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out ql/src/test/results/clientpositive/join0.q.out ql/src/test/results/clientpositive/join11.q.out ql/src/test/results/clientpositive/join12.q.out ql/src/test/results/clientpositive/join13.q.out ql/src/test/results/clientpositive/join14.q.out ql/src/test/results/clientpositive/join16.q.out ql/src/test/results/clientpositive/join20.q.out ql/src/test/results/clientpositive/join21.q.out ql/src/test/results/clientpositive/join23.q.out ql/src/test/results/clientpositive/join34.q.out
[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2249: -- Attachment: HIVE-2249.D1383.2.patch zhiqiu updated the revision HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double. Reviewers: njain, kevinwilfong, heyongqiang, JIRA, stuart983 * Address Kevin's comments: fix the failed test cases (insert1_overwrite_partitions.q, 2. insert2_overwrite_partitions.q 3. ppr_pushdown) * Add a unit test to verify the smart type inference works (infer_const_type.q) * Remove un-used lines and fix a bug that str_col = long_const does not return false REVISION DETAIL https://reviews.facebook.net/D1383 AFFECTED FILES contrib/src/test/results/clientpositive/dboutput.q.out contrib/src/test/results/clientpositive/serde_typedbytes4.q.out ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java ql/src/test/queries/clientpositive/infer_const_type.q ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q ql/src/test/queries/clientpositive/ppr_pushdown.q ql/src/test/results/clientpositive/auto_join0.q.out ql/src/test/results/clientpositive/auto_join11.q.out ql/src/test/results/clientpositive/auto_join12.q.out ql/src/test/results/clientpositive/auto_join13.q.out ql/src/test/results/clientpositive/auto_join14.q.out ql/src/test/results/clientpositive/auto_join16.q.out ql/src/test/results/clientpositive/auto_join20.q.out ql/src/test/results/clientpositive/auto_join21.q.out ql/src/test/results/clientpositive/auto_join23.q.out ql/src/test/results/clientpositive/auto_join27.q.out ql/src/test/results/clientpositive/auto_join28.q.out ql/src/test/results/clientpositive/auto_join29.q.out ql/src/test/results/clientpositive/auto_join4.q.out ql/src/test/results/clientpositive/auto_join5.q.out ql/src/test/results/clientpositive/auto_join6.q.out ql/src/test/results/clientpositive/auto_join7.q.out ql/src/test/results/clientpositive/auto_join8.q.out ql/src/test/results/clientpositive/cast1.q.out ql/src/test/results/clientpositive/cluster.q.out ql/src/test/results/clientpositive/create_view.q.out ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out ql/src/test/results/clientpositive/having.q.out ql/src/test/results/clientpositive/index_auto.q.out ql/src/test/results/clientpositive/index_auto_empty.q.out ql/src/test/results/clientpositive/index_auto_file_format.q.out ql/src/test/results/clientpositive/index_auto_mult_tables.q.out ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out ql/src/test/results/clientpositive/index_auto_multiple.q.out ql/src/test/results/clientpositive/index_auto_partitioned.q.out ql/src/test/results/clientpositive/index_auto_self_join.q.out ql/src/test/results/clientpositive/index_auto_unused.q.out ql/src/test/results/clientpositive/index_auto_update.q.out ql/src/test/results/clientpositive/index_bitmap3.q.out ql/src/test/results/clientpositive/index_bitmap_auto.q.out ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out ql/src/test/results/clientpositive/index_bitmap_compression.q.out ql/src/test/results/clientpositive/index_compression.q.out ql/src/test/results/clientpositive/index_stale.q.out ql/src/test/results/clientpositive/index_stale_partitioned.q.out ql/src/test/results/clientpositive/infer_const_type.q.out ql/src/test/results/clientpositive/input11.q.out ql/src/test/results/clientpositive/input11_limit.q.out ql/src/test/results/clientpositive/input12.q.out ql/src/test/results/clientpositive/input13.q.out ql/src/test/results/clientpositive/input14.q.out ql/src/test/results/clientpositive/input14_limit.q.out ql/src/test/results/clientpositive/input18.q.out ql/src/test/results/clientpositive/input1_limit.q.out ql/src/test/results/clientpositive/input2_limit.q.out ql/src/test/results/clientpositive/input42.q.out ql/src/test/results/clientpositive/input_part1.q.out ql/src/test/results/clientpositive/input_part2.q.out ql/src/test/results/clientpositive/input_part5.q.out ql/src/test/results/clientpositive/input_part7.q.out ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out ql/src/test/results/clientpositive/join0.q.out ql/src/test/results/clientpositive/join11.q.out ql/src/test/results/clientpositive/join12.q.out ql/src/test/results/clientpositive/join13.q.out ql/src/test/results/clientpositive/join14.q.out ql/src/test/results/clientpositive/join16.q.out ql/src/test/results/clientpositive/join20.q.out ql/src/test/results/clientpositive/join21.q.out
[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2249:
--
Attachment: HIVE-2249.D1383.1.patch
zhiqiu requested code review of HIVE-2249 [jira] When creating constant
expression for numbers, try to infer type from another comparison operand,
instead of trying to use integer first, and then long and double.
Reviewers: njain, kevinwilfong, heyongqiang, JIRA
[jira] [HIVE-2249] Smart type inference of constants
This patch adds support to smartly infer the constant's type when
encountering the query like column CMP constant, where CMP could be
any of the comparators supported by Hive. This aims to improving the
performance by moving the type conversion from runtime stage to
compiling stage.
To be more detailed, the smart type inference will happen when the type
of the column is one of the followings:
* TINYINT
* SMALLINT
* INT
* BIGINT
* FLOAT
* DOUBLE
If the type of the columns fits any of the above, the constant on the
other hand side will be converted firstly to the column's type.
* TINYINT = Byte
* SMALLINT = Short
* INT = Integer
* BIGINT = Long
* FLOAT = Float
* DOUBLE = Double
If failing, the constant will then be converted to DOUBLE. If both tries
fail, the constant will be left as what type it is.
One exception is when the column is STRING while the constant is BIGINT.
In this case, we do nothing. Otherwise, the constant will be converted
to DOUBLE.
Other improvements include returning false immediately for the query
like int_col = not_convertable_double_constant, such as uid = 1.5.
NOTE:
~130 unit test cases need to be updated due to this diff. All updates
are limited to convert to the plan like col = 10 to col = 10.0, and
are carefully checked individually.
TWO test cases failed during the unit testing:
* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown
When looking into the query as well as the output, the plans generated
were found to be the same while the query results changed. As the
queries in these two cases are simple select queries, maybe the default
sorting criteria was changed unintentionally by this diff or other
diffs.
Task ID: #620808
Blame Rev:
The current code to build constant expression for numbers, here is the code:
try {
v = Double.valueOf(expr.getText());
v = Long.valueOf(expr.getText());
v = Integer.valueOf(expr.getText());
} catch (NumberFormatException e) {
// do nothing here, we will throw an exception in the following block
}
if (v == null) {
throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
.getMsg(expr));
}
return new ExprNodeConstantDesc(v);
The for the case that WHERE BIG_INT_COLUMN = 0, or WHERE DOUBLE_COLUMN
= 0, we always have to do a type conversion when comparing, which is
unnecessary if it is slightly smarter to choose type when creating the constant
expression. We can simply walk one level up the tree, find another comparison
party and use the same type with that one if it is possible. For user's wrong
query like 'INT_COLUMN=1.1', we can even do more.
TEST PLAN
Run unit tests.
TWO test cases failed during the unit testing:
* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown
When looking into the query as well as the output, the plans generated
were found to be the same while the query results changed. As the
queries in these two cases are simple select queries, maybe the default
sorting criteria was changed unintentionally by this diff or other
diffs.
Revert Plan:
Tags:
REVISION DETAIL
https://reviews.facebook.net/D1383
AFFECTED FILES
contrib/src/test/results/clientpositive/dboutput.q.out
contrib/src/test/results/clientpositive/serde_typedbytes4.q.out
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
ql/src/test/results/clientpositive/auto_join0.q.out
ql/src/test/results/clientpositive/auto_join11.q.out
ql/src/test/results/clientpositive/auto_join12.q.out
ql/src/test/results/clientpositive/auto_join13.q.out
ql/src/test/results/clientpositive/auto_join14.q.out
ql/src/test/results/clientpositive/auto_join16.q.out
ql/src/test/results/clientpositive/auto_join20.q.out
ql/src/test/results/clientpositive/auto_join21.q.out
ql/src/test/results/clientpositive/auto_join23.q.out
ql/src/test/results/clientpositive/auto_join27.q.out
ql/src/test/results/clientpositive/auto_join28.q.out
ql/src/test/results/clientpositive/auto_join29.q.out
ql/src/test/results/clientpositive/auto_join4.q.out
ql/src/test/results/clientpositive/auto_join5.q.out
ql/src/test/results/clientpositive/auto_join6.q.out
ql/src/test/results/clientpositive/auto_join7.q.out
[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhiqiu Kong updated HIVE-2249:
--
Attachment: HIVE-2249.2.patch.txt
This patch adds support to smartly infer the constant's type when encountering
the query like column CMP constant, where CMP could be any of the comparators
supported by Hive. This aims to improving the performance by moving the type
conversion from runtime stage to compiling stage.
To be more detailed, the smart type inference will happen when the type of the
column is on e of the following:
* TINYINT
* SMALLINT
* INT
* BIGINT
* FLOAT
* DOUBLE
If the type of the columns fits any of the above, the constant on the other
hand side will be converted firstly to the column's type. When failing, the
constant will then be converted to DOUBLE. If both tries fail, the constant
will be left as what type it is.
One exception is when the column is STRING while the constant is BIGINT. In
this case, we do nothing. Otherwise, the constant will be converted to DOUBLE.
Other improvements include returning false immediately for the query like
int_col = not_convertable_double_constant, such as uid = 1.5.
NOTE:
~130 unit test cases need to be updated due to this diff. All updates are
limited to convert to the plan like col = 10 to col = 10.0, and are
carefully checked individually.
TWO test cases failed during the unit testing:
* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown
When looking into the query as well as the output, the plans generated were
found to be the same while the query results changed. As the queries in these
two cases are simple select queries, maybe the default sorting criteria was
changed unintentionally by this diff or other diffs.
When creating constant expression for numbers, try to infer type from another
comparison operand, instead of trying to use integer first, and then long and
double
--
Key: HIVE-2249
URL: https://issues.apache.org/jira/browse/HIVE-2249
Project: Hive
Issue Type: Improvement
Reporter: Siying Dong
Assignee: Joseph Barillari
Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt
The current code to build constant expression for numbers, here is the code:
try {
v = Double.valueOf(expr.getText());
v = Long.valueOf(expr.getText());
v = Integer.valueOf(expr.getText());
} catch (NumberFormatException e) {
// do nothing here, we will throw an exception in the following block
}
if (v == null) {
throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
.getMsg(expr));
}
return new ExprNodeConstantDesc(v);
The for the case that WHERE BIG_INT_COLUMN = 0, or WHERE DOUBLE_COLUMN
= 0, we always have to do a type conversion when comparing, which is
unnecessary if it is slightly smarter to choose type when creating the
constant expression. We can simply walk one level up the tree, find another
comparison party and use the same type with that one if it is possible. For
user's wrong query like 'INT_COLUMN=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Barillari updated HIVE-2249:
---
Attachment: HIVE-2249.1.patch.txt
As explained in the comments in TypeCheckProcFactory, this patch implements
this fix. It also optimizes impossible comparisons (int column and double
constant) to FALSE.
It also includes a new unit test, type_matching.q. Note that this test does
(and should) produce slightly different output between the trunk version and
the patched version, since it runs EXPLAIN SELECT on several queries that this
patch will modify, so the string outputs (e.g., 22 vs 22.0 are different.)
When creating constant expression for numbers, try to infer type from another
comparison operand, instead of trying to use integer first, and then long and
double
--
Key: HIVE-2249
URL: https://issues.apache.org/jira/browse/HIVE-2249
Project: Hive
Issue Type: Improvement
Reporter: Siying Dong
Attachments: HIVE-2249.1.patch.txt
The current code to build constant expression for numbers, here is the code:
try {
v = Double.valueOf(expr.getText());
v = Long.valueOf(expr.getText());
v = Integer.valueOf(expr.getText());
} catch (NumberFormatException e) {
// do nothing here, we will throw an exception in the following block
}
if (v == null) {
throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
.getMsg(expr));
}
return new ExprNodeConstantDesc(v);
The for the case that WHERE BIG_INT_COLUMN = 0, or WHERE DOUBLE_COLUMN
= 0, we always have to do a type conversion when comparing, which is
unnecessary if it is slightly smarter to choose type when creating the
constant expression. We can simply walk one level up the tree, find another
comparison party and use the same type with that one if it is possible. For
user's wrong query like 'INT_COLUMN=1.1', we can even do more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
