[jira] [Commented] (HIVE-15084) Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
[
https://issues.apache.org/jira/browse/HIVE-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787130#comment-15787130
]
Pengcheng Xiong commented on HIVE-15084:
L816 in GroupbyOperator:
{code}
if ( shouldBeFlushed(newKeys)) {
flushHashTable(false);
}
{code}
Sounds like not every key is flushed and it depends on the shouldBeFlushed
which depends on the system memory.
> Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
>
>
> Key: HIVE-15084
> URL: https://issues.apache.org/jira/browse/HIVE-15084
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Pengcheng Xiong
>
> Example diffs:
> {noformat:title=explainanalyze_2.q.out}
> 1881c1881
> < Group By Operator [GBY_2] (rows=1/500
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1
> > width=8)
> 2227c2227
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2237c2237
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> 2243c2243
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2260c2260
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> {noformat}
> {noformat:title=explainanalyze_2.q.out}
> 367c367
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 378c378
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 2135c2135
> < Group By Operator [GBY_2] (rows=1/241
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1 width=8)
> {noformat}
> From https://builds.apache.org/job/PreCommit-HIVE-Build/2295/testReport/:
> {noformat:title=explainanalyze_4.q.out}
> 248c248
> < Group By Operator [GBY_10] (rows=615/10 width=12)
> ---
> > Group By Operator [GBY_10] (rows=615/5 width=12)
> {noformat}
> {noformat:title=explainanalyze_5.q.out}
> 143c143
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> 154c154
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15112) Implement Parquet vectorization reader for Struct type
[ https://issues.apache.org/jira/browse/HIVE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-15112: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to the master. Thanks [~csun] for the review. > Implement Parquet vectorization reader for Struct type > -- > > Key: HIVE-15112 > URL: https://issues.apache.org/jira/browse/HIVE-15112 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Fix For: 2.2.0 > > Attachments: HIVE-15112.1.patch, HIVE-15112.2.patch, > HIVE-15112.3.patch, HIVE-15112.4.patch, HIVE-15112.5.patch, HIVE-15112.patch > > > Like HIVE-14815, we need support Parquet vectorized reader for struct type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15525) Hooking ChangeManager to "drop table", "drop partition"
[
https://issues.apache.org/jira/browse/HIVE-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786865#comment-15786865
]
Hive QA commented on HIVE-15525:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845141/HIVE-15525.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 10871 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=144)
[vectorized_rcfile_columnar.q,vector_elt.q,explainuser_1.q,multi_insert.q,tez_dml.q,vector_bround.q,schema_evol_orc_acid_table.q,vector_when_case_null.q,orc_ppd_schema_evol_1b.q,vector_join30.q,vectorization_11.q,cte_3.q,update_tmp_table.q,vector_decimal_cast.q,groupby_grouping_id2.q,vector_decimal_round.q,tez_smb_empty.q,orc_merge6.q,vector_char_mapjoin1.q,vector_decimal_trailing.q,cte_5.q,tez_union.q,vector_decimal_2.q,columnStatsUpdateForStatsOptimizer_1.q,vector_outer_join3.q,schema_evol_text_vec_part_all_complex.q,tez_dynpart_hashjoin_2.q,auto_sortmerge_join_12.q,offset_limit.q,tez_union_multiinsert.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=92)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join_without_localtask]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_3]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket_map_join_spark4]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin4]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[date_join1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby2_noskew]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby6_noskew]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[mapjoin_test_outer]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[merge2] (batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[multi_join_union]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_11]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[timestamp_2]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union21]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_9]
(batchId=94)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2750/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2750/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2750/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845141 - PreCommit-HIVE-Build
> Hooking ChangeManager to "drop table", "drop partition"
> ---
>
> Key: HIVE-15525
> URL: https://issues.apache.org/jira/browse/HIVE-15525
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-15525.1.patch
>
>
> When Hive "drop table"/"drop partition", we will move data files into cmroot
> in case the replication destination will need it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15517) NOT (x <=> y) returns NULL if x or y is NULL
[
https://issues.apache.org/jira/browse/HIVE-15517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786811#comment-15786811
]
Fei Hui commented on HIVE-15517:
reappear this problem on hive 2.0.0, i will dig it
> NOT (x <=> y) returns NULL if x or y is NULL
>
>
> Key: HIVE-15517
> URL: https://issues.apache.org/jira/browse/HIVE-15517
> Project: Hive
> Issue Type: Bug
> Components: Hive, Operators, Query Processor, SQL
>Affects Versions: 1.2.1
>Reporter: Alexey Bedrintsev
>
> I created a table as following:
> create table test(x string, y string);
> insert into test values ('q', 'q'), ('q', 'w'), (NULL, 'q'), ('q', NULL),
> (NULL, NULL);
> Then I try to compare values taking NULLs into account:
> select *, x<=>y, not (x<=> y), (x <=> y) = false from test;
> OK
> q q truefalse false
> q w false truetrue
> q NULLfalse NULLtrue
> NULLq false NULLtrue
> NULLNULLtrueNULLfalse
> I expected that 4th column will be the same as 5th one but actually got NULL
> as result of "not false" and "not true" expressions.
> Hive 1.2.1000.2.5.0.0-1245
> Subversion
> git://c66-slave-20176e25-3/grid/0/jenkins/workspace/HDP-parallel-centos6/SOURCES/hive
> -r da6c690d384d1666f5a5f450be5cbc54e2fe4bd6
> Compiled by jenkins on Fri Aug 26 01:39:52 UTC 2016
> From source with checksum c30648316a632f7a753f4359e5c8f4d6
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15474) Extend limit propagation for chain of RS-GB-RS operators
[
https://issues.apache.org/jira/browse/HIVE-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786788#comment-15786788
]
Rui Li commented on HIVE-15474:
---
[~jcamachorodriguez], thanks very much for the detailed explanations :) For
Spark, the operator chain is something like this:
{{GBY1 – RS2 – GBY3 – RS4 – SEL5 – FS6}}
Since RS2 can produce the top N keys, I think this optimization doesn't require
the input to GBY3 to be sorted. I mean we still feed the top N keys to GBY3,
but after shuffling, those keys may not be in a sorted order. And the result
should remain correct. Is that right?
> Extend limit propagation for chain of RS-GB-RS operators
>
>
> Key: HIVE-15474
> URL: https://issues.apache.org/jira/browse/HIVE-15474
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-15474.patch
>
>
> The goal is to extend the work started in HIVE-14002.
> For instance, given the following query:
> {code:sql}
> explain
> select key, value, count(key + 1) as agg1 from src
> group by key, value
> order by key, value, agg1 limit 20;
> {code}
> We generate the following physical plan:
> {{TS1 - GBY2 - RS3 - GBY4 - RS5 - SEL6 - LIM7 - FS8}}
> We can push the limit to RS3 operator, as we will generate records for the
> _top N_ keys, and thus, GBY4 will produce the _top N_ results. However,
> currently we do not do it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15112) Implement Parquet vectorization reader for Struct type
[
https://issues.apache.org/jira/browse/HIVE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786761#comment-15786761
]
Hive QA commented on HIVE-15112:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845134/HIVE-15112.5.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10913 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=92)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=92)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2749/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2749/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2749/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845134 - PreCommit-HIVE-Build
> Implement Parquet vectorization reader for Struct type
> --
>
> Key: HIVE-15112
> URL: https://issues.apache.org/jira/browse/HIVE-15112
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ferdinand Xu
>Assignee: Ferdinand Xu
> Fix For: 2.2.0
>
> Attachments: HIVE-15112.1.patch, HIVE-15112.2.patch,
> HIVE-15112.3.patch, HIVE-15112.4.patch, HIVE-15112.5.patch, HIVE-15112.patch
>
>
> Like HIVE-14815, we need support Parquet vectorized reader for struct type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15519) Hive Decimal Type column scale is returning as zero
[
https://issues.apache.org/jira/browse/HIVE-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786741#comment-15786741
]
Rui Li commented on HIVE-15519:
---
The failures are not related because I can't reproduce them locally.
[~thejas], please take a look. Thanks!
> Hive Decimal Type column scale is returning as zero
> ---
>
> Key: HIVE-15519
> URL: https://issues.apache.org/jira/browse/HIVE-15519
> Project: Hive
> Issue Type: Bug
> Components: Hive, JDBC
>Reporter: Bharat Viswanadham
>Assignee: Rui Li
>Priority: Critical
> Attachments: HIVE-15519.1.patch, HIVE-15519.2.patch,
> data_type_test(1).txt
>
>
> Hive decimal type column precision is returning as zero, even though column
> has precision set.
> Example: col67 decimal(18,2) scale is returning as zero for that column.
> Tried with below program.
> {code}
>System.out.println("Opening connection");
> Class.forName("org.apache.hive.jdbc.HiveDriver");
>Connection con =
> DriverManager.getConnection("jdbc:hive2://x.x.x.x:1/default");
> DatabaseMetaData dbMeta = con.getMetaData();
>ResultSet rs = dbMeta.getColumns(null, "DEFAULT", "data_type_test",null);
> while (rs.next()) {
> if (rs.getString("COLUMN_NAME").equalsIgnoreCase("col48") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col67") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col68") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col122")){
> System.out.println(rs.getString("COLUMN_NAME") + "\t" +
> rs.getString("COLUMN_SIZE") + "\t" + rs.getInt("DECIMAL_DIGITS"));
> }
>}
>rs.close();
>con.close();
> } catch (Exception e) {
>e.printStackTrace();
>;
> }
> {code}
> Default fetch size is 50. if any column no is under 50 with decimal type,
> precision is returning properly, when the column no is greater than 50, scale
> is returning as zero.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15525) Hooking ChangeManager to "drop table", "drop partition"
[ https://issues.apache.org/jira/browse/HIVE-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-15525: -- Status: Patch Available (was: Open) > Hooking ChangeManager to "drop table", "drop partition" > --- > > Key: HIVE-15525 > URL: https://issues.apache.org/jira/browse/HIVE-15525 > Project: Hive > Issue Type: Sub-task > Components: repl >Reporter: Daniel Dai >Assignee: Daniel Dai > Attachments: HIVE-15525.1.patch > > > When Hive "drop table"/"drop partition", we will move data files into cmroot > in case the replication destination will need it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15481) Support multiple and nested subqueries
[
https://issues.apache.org/jira/browse/HIVE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786687#comment-15786687
]
Hive QA commented on HIVE-15481:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845133/HIVE-15481.3.patch
{color:green}SUCCESS:{color} +1 due to 13 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10862 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=100)
[avro_joins.q,skewjoinopt16.q,auto_join14.q,vectorization_14.q,auto_join26.q,stats1.q,cbo_stats.q,auto_sortmerge_join_6.q,union22.q,union_remove_24.q,union_view.q,smb_mapjoin_22.q,stats15.q,ptf_matchpath.q,transform_ppr1.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=124)
[table_access_keys_stats.q,bucketmapjoin11.q,auto_join4.q,join34.q,nullgroup.q,mergejoins_mixed.q,sort.q,join_nullsafe.q,stats8.q,auto_join28.q,join17.q,union17.q,skewjoinopt11.q,groupby1_map.q,load_dyn_part11.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=103)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2748/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2748/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2748/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845133 - PreCommit-HIVE-Build
> Support multiple and nested subqueries
> --
>
> Key: HIVE-15481
> URL: https://issues.apache.org/jira/browse/HIVE-15481
> Project: Hive
> Issue Type: Sub-task
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-15481.1.patch, HIVE-15481.2.patch,
> HIVE-15481.3.patch
>
>
> This is continuation of the work done in HIVE-15192. As listed at
> [Restrictions |
> https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf ]
> currently it is not possible to execute queries which either have more than
> one subquery or have nested subquery.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15525) Hooking ChangeManager to "drop table", "drop partition"
[ https://issues.apache.org/jira/browse/HIVE-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-15525: -- Attachment: HIVE-15525.1.patch > Hooking ChangeManager to "drop table", "drop partition" > --- > > Key: HIVE-15525 > URL: https://issues.apache.org/jira/browse/HIVE-15525 > Project: Hive > Issue Type: Sub-task > Components: repl >Reporter: Daniel Dai >Assignee: Daniel Dai > Attachments: HIVE-15525.1.patch > > > When Hive "drop table"/"drop partition", we will move data files into cmroot > in case the replication destination will need it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15112) Implement Parquet vectorization reader for Struct type
[ https://issues.apache.org/jira/browse/HIVE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-15112: Attachment: HIVE-15112.5.patch Attach the same patch as 4th to see any new failed test cases. > Implement Parquet vectorization reader for Struct type > -- > > Key: HIVE-15112 > URL: https://issues.apache.org/jira/browse/HIVE-15112 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Fix For: 2.2.0 > > Attachments: HIVE-15112.1.patch, HIVE-15112.2.patch, > HIVE-15112.3.patch, HIVE-15112.4.patch, HIVE-15112.5.patch, HIVE-15112.patch > > > Like HIVE-14815, we need support Parquet vectorized reader for struct type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15507) Nested column pruning: fix issue when selecting struct field from array/map element
[
https://issues.apache.org/jira/browse/HIVE-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786564#comment-15786564
]
Ferdinand Xu commented on HIVE-15507:
-
Sorry for my late response. Thanks [~csun] for the patch. LGTM +1
> Nested column pruning: fix issue when selecting struct field from array/map
> element
> ---
>
> Key: HIVE-15507
> URL: https://issues.apache.org/jira/browse/HIVE-15507
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer, Physical Optimizer,
> Serializers/Deserializers
>Affects Versions: 2.2.0
>Reporter: Chao Sun
>Assignee: Chao Sun
> Attachments: 15507.1.patch
>
>
> When running the following query:
> {code}
> SELECT count(col), arr[0].f
> FROM tbl
> GROUP BY arr[0].f
> {code}
> where {{arr}} is an array of struct with field {{f}}. Nested column pruning
> will fail. This is because we currently process {{GenericUDFIndex}} in the
> same way as any other UDF. In this case, it will generate path {{arr.f}},
> which will not match the struct type info when doing the pruning.
> Same thing for map.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15481) Support multiple and nested subqueries
[ https://issues.apache.org/jira/browse/HIVE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-15481: --- Attachment: HIVE-15481.3.patch > Support multiple and nested subqueries > -- > > Key: HIVE-15481 > URL: https://issues.apache.org/jira/browse/HIVE-15481 > Project: Hive > Issue Type: Sub-task > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-15481.1.patch, HIVE-15481.2.patch, > HIVE-15481.3.patch > > > This is continuation of the work done in HIVE-15192. As listed at > [Restrictions | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf ] > currently it is not possible to execute queries which either have more than > one subquery or have nested subquery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15481) Support multiple and nested subqueries
[ https://issues.apache.org/jira/browse/HIVE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-15481: --- Status: Patch Available (was: Open) > Support multiple and nested subqueries > -- > > Key: HIVE-15481 > URL: https://issues.apache.org/jira/browse/HIVE-15481 > Project: Hive > Issue Type: Sub-task > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-15481.1.patch, HIVE-15481.2.patch, > HIVE-15481.3.patch > > > This is continuation of the work done in HIVE-15192. As listed at > [Restrictions | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf ] > currently it is not possible to execute queries which either have more than > one subquery or have nested subquery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15481) Support multiple and nested subqueries
[ https://issues.apache.org/jira/browse/HIVE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-15481: --- Status: Open (was: Patch Available) > Support multiple and nested subqueries > -- > > Key: HIVE-15481 > URL: https://issues.apache.org/jira/browse/HIVE-15481 > Project: Hive > Issue Type: Sub-task > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-15481.1.patch, HIVE-15481.2.patch > > > This is continuation of the work done in HIVE-15192. As listed at > [Restrictions | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf ] > currently it is not possible to execute queries which either have more than > one subquery or have nested subquery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15437) avro tables join fails when - tbl join tbl_postfix
[
https://issues.apache.org/jira/browse/HIVE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-15437:
-
Description:
The following queries return good results:
select * from table1 where col1=key1;
select * from table1_1 where col1=key1;
When join them together, it gets following error:
{noformat}
Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found long,
expecting union
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:43)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:141)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
{noformat}
The two avro tables both is defined by using avro schema, and the first table's
name is the second table name's prefix.
Note that this happens when single map is reading input from both tables. If
map-join is used with MR execution engine, or if Tez execution engine is used,
this issue is not seen.
was:
The following queries return good results:
select * from table1 where col1=key1;
select * from table1_1 where col1=key1;
When join them together, it gets following error:
{noformat}
Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found long,
expecting union
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:43)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:141)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
{noformat}
The two avro tables both is defined by using avro schema, and the first table's
name is the second table name's prefix.
> avro tables join fails when - tbl join tbl_postfix
> --
>
> Key: HIVE-15437
> URL: https://issues.apache.org/jira/browse/HIVE-15437
> Project: Hive
> Issue Type: Bug
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
> Fix For: 2.2.0
>
> Attachments: HIVE-15437.1.patch
>
>
> The following queries return good results:
> select * from table1 where col1=key1;
> select * from table1_1 where col1=key1;
> When join them together, it gets following error:
> {noformat}
> Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found
> long, expecting union
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
> ~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextEx
[jira] [Updated] (HIVE-13557) Make interval keyword optional while specifying DAY in interval arithmetic
[
https://issues.apache.org/jira/browse/HIVE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-13557:
---
Affects Version/s: 2.1.0
> Make interval keyword optional while specifying DAY in interval arithmetic
> --
>
> Key: HIVE-13557
> URL: https://issues.apache.org/jira/browse/HIVE-13557
> Project: Hive
> Issue Type: Sub-task
> Components: Types
>Affects Versions: 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Zoltan Haindrich
> Fix For: 2.2.0
>
> Attachments: HIVE-13557.1.patch, HIVE-13557.1.patch,
> HIVE-13557.1.patch, HIVE-13557.2.patch, HIVE-13557.3.patch, HIVE-13557.3.patch
>
>
> Currently we support expressions like: {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) - INTERVAL '30' DAY) AND
> DATE('2000-01-31')
> {code}
> We should support:
> {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) + (-30) DAY) AND
> DATE('2000-01-31')
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13557) Make interval keyword optional while specifying DAY in interval arithmetic
[
https://issues.apache.org/jira/browse/HIVE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-13557:
---
Fix Version/s: 2.2.0
> Make interval keyword optional while specifying DAY in interval arithmetic
> --
>
> Key: HIVE-13557
> URL: https://issues.apache.org/jira/browse/HIVE-13557
> Project: Hive
> Issue Type: Sub-task
> Components: Types
>Affects Versions: 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Zoltan Haindrich
> Fix For: 2.2.0
>
> Attachments: HIVE-13557.1.patch, HIVE-13557.1.patch,
> HIVE-13557.1.patch, HIVE-13557.2.patch, HIVE-13557.3.patch, HIVE-13557.3.patch
>
>
> Currently we support expressions like: {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) - INTERVAL '30' DAY) AND
> DATE('2000-01-31')
> {code}
> We should support:
> {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) + (-30) DAY) AND
> DATE('2000-01-31')
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15522) REPL LOAD & DUMP support for incremental ALTER_TABLE/ALTER_PTN including renames
[ https://issues.apache.org/jira/browse/HIVE-15522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786204#comment-15786204 ] Thejas M Nair commented on HIVE-15522: -- Added some minor comments in review board. Looks great overall. Its good that some of the refactoring/cleanup has been done in this patch, that should reduce the overall cleanup work and also structure new patches in the cleaner way. > REPL LOAD & DUMP support for incremental ALTER_TABLE/ALTER_PTN including > renames > > > Key: HIVE-15522 > URL: https://issues.apache.org/jira/browse/HIVE-15522 > Project: Hive > Issue Type: Sub-task > Components: repl >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Attachments: HIVE-15522.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15518) Refactoring rows and range related classes to put the window type on Window
[
https://issues.apache.org/jira/browse/HIVE-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786162#comment-15786162
]
Hive QA commented on HIVE-15518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845096/HIVE-15518.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10869 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=144)
[vectorized_rcfile_columnar.q,vector_elt.q,explainuser_1.q,multi_insert.q,tez_dml.q,vector_bround.q,schema_evol_orc_acid_table.q,vector_when_case_null.q,orc_ppd_schema_evol_1b.q,vector_join30.q,vectorization_11.q,cte_3.q,update_tmp_table.q,vector_decimal_cast.q,groupby_grouping_id2.q,vector_decimal_round.q,tez_smb_empty.q,orc_merge6.q,vector_char_mapjoin1.q,vector_decimal_trailing.q,cte_5.q,tez_union.q,vector_decimal_2.q,columnStatsUpdateForStatsOptimizer_1.q,vector_outer_join3.q,schema_evol_text_vec_part_all_complex.q,tez_dynpart_hashjoin_2.q,auto_sortmerge_join_12.q,offset_limit.q,tez_union_multiinsert.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1]
(batchId=92)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=92)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2747/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2747/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2747/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845096 - PreCommit-HIVE-Build
> Refactoring rows and range related classes to put the window type on Window
> ---
>
> Key: HIVE-15518
> URL: https://issues.apache.org/jira/browse/HIVE-15518
> Project: Hive
> Issue Type: Sub-task
> Components: PTF-Windowing
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15518.1.patch, HIVE-15518.2.patch,
> HIVE-15518.3.patch, HIVE-15518.4.patch
>
>
> {noformat}
> /*
>* - A Window Frame that has only the /start/boundary, then it is
> interpreted as:
> BETWEEN AND CURRENT ROW
>* - A Window Specification with an Order Specification and no Window
>* Frame is interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
>* - A Window Specification with no Order and no Window Frame is
> interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
>*/
> {noformat}
> The comments in WindowSpec above doesn't really match what it's claimed to
> do. Correct the comment to reduce the confusion.
> Also currently the window type is specified on each BoundarySpec but makes
> sense to put the type (rows or range) for each window.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15493) Wrong result for LEFT outer join in Tez using MapJoinOperator
[
https://issues.apache.org/jira/browse/HIVE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-15493:
---
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks for reviewing [~pxiong]!
I have created HIVE-15524 to tackle the root cause of the issue.
> Wrong result for LEFT outer join in Tez using MapJoinOperator
> -
>
> Key: HIVE-15493
> URL: https://issues.apache.org/jira/browse/HIVE-15493
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HIVE-15493.01.patch, HIVE-15493.patch
>
>
> To reproduce, we can run in Tez:
> {code:sql}
> set hive.auto.convert.join=true;
> DROP TABLE IF EXISTS test_1;
> CREATE TABLE test_1
> (
> member BIGINT
> , age VARCHAR (100)
> )
> STORED AS TEXTFILE
> ;
> DROP TABLE IF EXISTS test_2;
> CREATE TABLE test_2
> (
> member BIGINT
> )
> STORED AS TEXTFILE
> ;
> INSERT INTO test_1 VALUES (1, '20'), (2, '30'), (3, '40');
> INSERT INTO test_2 VALUES (1), (2), (3);
> SELECT
> t2.member
> , t1.age_1
> , t1.age_2
> FROM
> test_2 t2
> LEFT JOIN (
> SELECT
> member
> , age as age_1
> , age as age_2
> FROM
> test_1
> ) t1
> ON t2.member = t1.member
> ;
> {code}
> Result is:
> {noformat}
> 1 20 NULL
> 3 40 NULL
> 2 30 NULL
> {noformat}
> Correct result is:
> {noformat}
> 1 20 20
> 3 40 40
> 2 30 30
> {noformat}
> Bug was introduced by HIVE-10582. Though the fix in HIVE-10582 does not
> contain tests, it does look legit. In fact, the problem seems to be in the
> MapJoinOperator itself. It only happens for LEFT outer join (not with RIGHT
> outer or FULL outer). Although I am still trying to understand part of the
> MapJoinOperator code path, the bug could be in the initialization of the
> operator. It only happens when we have duplicate values in the right part of
> the output.
> Till we have more time to study the problem in detail and fix the
> MapJoinOperator, I will submit a fix that removes the code in
> SemanticAnalyzer that reuses duplicated value expressions from RS to create
> multiple columns in the join output (this is equivalent to reverting
> HIVE-10582).
> Once this is pushed, I will create a follow-up issue to take this code back
> and tackle the problem in the MapJoinOperator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15518) Refactoring rows and range related classes to put the window type on Window
[
https://issues.apache.org/jira/browse/HIVE-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-15518:
Attachment: HIVE-15518.4.patch
patch-4: fix additional unit tests.
> Refactoring rows and range related classes to put the window type on Window
> ---
>
> Key: HIVE-15518
> URL: https://issues.apache.org/jira/browse/HIVE-15518
> Project: Hive
> Issue Type: Sub-task
> Components: PTF-Windowing
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15518.1.patch, HIVE-15518.2.patch,
> HIVE-15518.3.patch, HIVE-15518.4.patch
>
>
> {noformat}
> /*
>* - A Window Frame that has only the /start/boundary, then it is
> interpreted as:
> BETWEEN AND CURRENT ROW
>* - A Window Specification with an Order Specification and no Window
>* Frame is interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
>* - A Window Specification with no Order and no Window Frame is
> interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
>*/
> {noformat}
> The comments in WindowSpec above doesn't really match what it's claimed to
> do. Correct the comment to reduce the confusion.
> Also currently the window type is specified on each BoundarySpec but makes
> sense to put the type (rows or range) for each window.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15493) Wrong result for LEFT outer join in Tez using MapJoinOperator
[
https://issues.apache.org/jira/browse/HIVE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785948#comment-15785948
]
Pengcheng Xiong commented on HIVE-15493:
I see. It makes more sense now... +1. :)
> Wrong result for LEFT outer join in Tez using MapJoinOperator
> -
>
> Key: HIVE-15493
> URL: https://issues.apache.org/jira/browse/HIVE-15493
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-15493.01.patch, HIVE-15493.patch
>
>
> To reproduce, we can run in Tez:
> {code:sql}
> set hive.auto.convert.join=true;
> DROP TABLE IF EXISTS test_1;
> CREATE TABLE test_1
> (
> member BIGINT
> , age VARCHAR (100)
> )
> STORED AS TEXTFILE
> ;
> DROP TABLE IF EXISTS test_2;
> CREATE TABLE test_2
> (
> member BIGINT
> )
> STORED AS TEXTFILE
> ;
> INSERT INTO test_1 VALUES (1, '20'), (2, '30'), (3, '40');
> INSERT INTO test_2 VALUES (1), (2), (3);
> SELECT
> t2.member
> , t1.age_1
> , t1.age_2
> FROM
> test_2 t2
> LEFT JOIN (
> SELECT
> member
> , age as age_1
> , age as age_2
> FROM
> test_1
> ) t1
> ON t2.member = t1.member
> ;
> {code}
> Result is:
> {noformat}
> 1 20 NULL
> 3 40 NULL
> 2 30 NULL
> {noformat}
> Correct result is:
> {noformat}
> 1 20 20
> 3 40 40
> 2 30 30
> {noformat}
> Bug was introduced by HIVE-10582. Though the fix in HIVE-10582 does not
> contain tests, it does look legit. In fact, the problem seems to be in the
> MapJoinOperator itself. It only happens for LEFT outer join (not with RIGHT
> outer or FULL outer). Although I am still trying to understand part of the
> MapJoinOperator code path, the bug could be in the initialization of the
> operator. It only happens when we have duplicate values in the right part of
> the output.
> Till we have more time to study the problem in detail and fix the
> MapJoinOperator, I will submit a fix that removes the code in
> SemanticAnalyzer that reuses duplicated value expressions from RS to create
> multiple columns in the join output (this is equivalent to reverting
> HIVE-10582).
> Once this is pushed, I will create a follow-up issue to take this code back
> and tackle the problem in the MapJoinOperator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14870) OracleStore: RawStore implementation optimized for Oracle
[ https://issues.apache.org/jira/browse/HIVE-14870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785946#comment-15785946 ] Chris Drome commented on HIVE-14870: [~alangates], I have attached a patch and schema for the work that I've completed to date. The patch is relative to our internal version of branch-1.2.1. There are a number of methods that are specific to our implementation of data discovery. The schema should be self-explanatory and adds tables required by the OracleStore implementation. I'm currently investigating some random deadlock exceptions that Oracle throws from time to time when dropping databases/tables on sparsely populated Oracle tables under high concurrency. I'll likely have to remove some of the cascaded deletes on foreign keys (or implement a retry on drop database/table) to address this issue. > OracleStore: RawStore implementation optimized for Oracle > - > > Key: HIVE-14870 > URL: https://issues.apache.org/jira/browse/HIVE-14870 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Chris Drome >Assignee: Chris Drome > Attachments: HIVE-14870.patch, OracleStoreDesignProposal.pdf, > schema-oraclestore.sql > > > The attached document is a proposal for a RawStore implementation which is > optimized for Oracle and replaces DataNucleus. The document outlines schema > changes, OracleStore implementation details, and performance tests against > ObjectStore, ObjectStore+DirectSQL, and OracleStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14870) OracleStore: RawStore implementation optimized for Oracle
[ https://issues.apache.org/jira/browse/HIVE-14870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-14870: --- Attachment: schema-oraclestore.sql HIVE-14870.patch > OracleStore: RawStore implementation optimized for Oracle > - > > Key: HIVE-14870 > URL: https://issues.apache.org/jira/browse/HIVE-14870 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Chris Drome >Assignee: Chris Drome > Attachments: HIVE-14870.patch, OracleStoreDesignProposal.pdf, > schema-oraclestore.sql > > > The attached document is a proposal for a RawStore implementation which is > optimized for Oracle and replaces DataNucleus. The document outlines schema > changes, OracleStore implementation details, and performance tests against > ObjectStore, ObjectStore+DirectSQL, and OracleStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15493) Wrong result for LEFT outer join in Tez using MapJoinOperator
[
https://issues.apache.org/jira/browse/HIVE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785916#comment-15785916
]
Jesus Camacho Rodriguez commented on HIVE-15493:
[~pxiong], the basic idea behind that code (HIVE-13191 was just an extension)
was that if we have multiple values in one of the join inputs that are equal,
we do not need to have them in the RS operator multiple times (and thus shuffle
them). Instead, we can have the value only once in the RS operator, and then
the join will read _x_ times that same value to produce the correct output. In
fact, the code was not working properly originally till HIVE-10582 went in.
The problem is that MapJoinOperator does not support duplicate values for left
outer join properly: there is an assumption for the row container about the
join output columns being the same as the input columns. I have not had the
chance to check that code in detail. Till then, this fix will avoid producing
incorrect results by not reusing the value in the RS and thus producing it
multiple times.
I will add a TODO to the code, and as I said in the issue description, I will
create a follow-up issue to tackle the root cause of the problem.
> Wrong result for LEFT outer join in Tez using MapJoinOperator
> -
>
> Key: HIVE-15493
> URL: https://issues.apache.org/jira/browse/HIVE-15493
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-15493.01.patch, HIVE-15493.patch
>
>
> To reproduce, we can run in Tez:
> {code:sql}
> set hive.auto.convert.join=true;
> DROP TABLE IF EXISTS test_1;
> CREATE TABLE test_1
> (
> member BIGINT
> , age VARCHAR (100)
> )
> STORED AS TEXTFILE
> ;
> DROP TABLE IF EXISTS test_2;
> CREATE TABLE test_2
> (
> member BIGINT
> )
> STORED AS TEXTFILE
> ;
> INSERT INTO test_1 VALUES (1, '20'), (2, '30'), (3, '40');
> INSERT INTO test_2 VALUES (1), (2), (3);
> SELECT
> t2.member
> , t1.age_1
> , t1.age_2
> FROM
> test_2 t2
> LEFT JOIN (
> SELECT
> member
> , age as age_1
> , age as age_2
> FROM
> test_1
> ) t1
> ON t2.member = t1.member
> ;
> {code}
> Result is:
> {noformat}
> 1 20 NULL
> 3 40 NULL
> 2 30 NULL
> {noformat}
> Correct result is:
> {noformat}
> 1 20 20
> 3 40 40
> 2 30 30
> {noformat}
> Bug was introduced by HIVE-10582. Though the fix in HIVE-10582 does not
> contain tests, it does look legit. In fact, the problem seems to be in the
> MapJoinOperator itself. It only happens for LEFT outer join (not with RIGHT
> outer or FULL outer). Although I am still trying to understand part of the
> MapJoinOperator code path, the bug could be in the initialization of the
> operator. It only happens when we have duplicate values in the right part of
> the output.
> Till we have more time to study the problem in detail and fix the
> MapJoinOperator, I will submit a fix that removes the code in
> SemanticAnalyzer that reuses duplicated value expressions from RS to create
> multiple columns in the join output (this is equivalent to reverting
> HIVE-10582).
> Once this is pushed, I will create a follow-up issue to take this code back
> and tackle the problem in the MapJoinOperator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15493) Wrong result for LEFT outer join in Tez using MapJoinOperator
[
https://issues.apache.org/jira/browse/HIVE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785825#comment-15785825
]
Pengcheng Xiong commented on HIVE-15493:
[~jcamachorodriguez], the patch sounds like a magic to me... :) I am wondering
why we have those redundant code in the past. And when I traced back, it was
generated in HIVE-13191 by yourself. :) Could you put some comments in the jira
explaining why the code works now? Thanks.
> Wrong result for LEFT outer join in Tez using MapJoinOperator
> -
>
> Key: HIVE-15493
> URL: https://issues.apache.org/jira/browse/HIVE-15493
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-15493.01.patch, HIVE-15493.patch
>
>
> To reproduce, we can run in Tez:
> {code:sql}
> set hive.auto.convert.join=true;
> DROP TABLE IF EXISTS test_1;
> CREATE TABLE test_1
> (
> member BIGINT
> , age VARCHAR (100)
> )
> STORED AS TEXTFILE
> ;
> DROP TABLE IF EXISTS test_2;
> CREATE TABLE test_2
> (
> member BIGINT
> )
> STORED AS TEXTFILE
> ;
> INSERT INTO test_1 VALUES (1, '20'), (2, '30'), (3, '40');
> INSERT INTO test_2 VALUES (1), (2), (3);
> SELECT
> t2.member
> , t1.age_1
> , t1.age_2
> FROM
> test_2 t2
> LEFT JOIN (
> SELECT
> member
> , age as age_1
> , age as age_2
> FROM
> test_1
> ) t1
> ON t2.member = t1.member
> ;
> {code}
> Result is:
> {noformat}
> 1 20 NULL
> 3 40 NULL
> 2 30 NULL
> {noformat}
> Correct result is:
> {noformat}
> 1 20 20
> 3 40 40
> 2 30 30
> {noformat}
> Bug was introduced by HIVE-10582. Though the fix in HIVE-10582 does not
> contain tests, it does look legit. In fact, the problem seems to be in the
> MapJoinOperator itself. It only happens for LEFT outer join (not with RIGHT
> outer or FULL outer). Although I am still trying to understand part of the
> MapJoinOperator code path, the bug could be in the initialization of the
> operator. It only happens when we have duplicate values in the right part of
> the output.
> Till we have more time to study the problem in detail and fix the
> MapJoinOperator, I will submit a fix that removes the code in
> SemanticAnalyzer that reuses duplicated value expressions from RS to create
> multiple columns in the join output (this is equivalent to reverting
> HIVE-10582).
> Once this is pushed, I will create a follow-up issue to take this code back
> and tackle the problem in the MapJoinOperator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-15084) Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
[
https://issues.apache.org/jira/browse/HIVE-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong reassigned HIVE-15084:
--
Assignee: Pengcheng Xiong
> Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
>
>
> Key: HIVE-15084
> URL: https://issues.apache.org/jira/browse/HIVE-15084
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Pengcheng Xiong
>
> Example diffs:
> {noformat:title=explainanalyze_2.q.out}
> 1881c1881
> < Group By Operator [GBY_2] (rows=1/500
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1
> > width=8)
> 2227c2227
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2237c2237
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> 2243c2243
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2260c2260
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> {noformat}
> {noformat:title=explainanalyze_2.q.out}
> 367c367
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 378c378
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 2135c2135
> < Group By Operator [GBY_2] (rows=1/241
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1 width=8)
> {noformat}
> From https://builds.apache.org/job/PreCommit-HIVE-Build/2295/testReport/:
> {noformat:title=explainanalyze_4.q.out}
> 248c248
> < Group By Operator [GBY_10] (rows=615/10 width=12)
> ---
> > Group By Operator [GBY_10] (rows=615/5 width=12)
> {noformat}
> {noformat:title=explainanalyze_5.q.out}
> 143c143
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> 154c154
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15084) Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
[
https://issues.apache.org/jira/browse/HIVE-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785808#comment-15785808
]
Pengcheng Xiong commented on HIVE-15084:
I will take a look today.
> Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
>
>
> Key: HIVE-15084
> URL: https://issues.apache.org/jira/browse/HIVE-15084
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Pengcheng Xiong
>
> Example diffs:
> {noformat:title=explainanalyze_2.q.out}
> 1881c1881
> < Group By Operator [GBY_2] (rows=1/500
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1
> > width=8)
> 2227c2227
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2237c2237
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> 2243c2243
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2260c2260
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> {noformat}
> {noformat:title=explainanalyze_2.q.out}
> 367c367
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 378c378
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 2135c2135
> < Group By Operator [GBY_2] (rows=1/241
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1 width=8)
> {noformat}
> From https://builds.apache.org/job/PreCommit-HIVE-Build/2295/testReport/:
> {noformat:title=explainanalyze_4.q.out}
> 248c248
> < Group By Operator [GBY_10] (rows=615/10 width=12)
> ---
> > Group By Operator [GBY_10] (rows=615/5 width=12)
> {noformat}
> {noformat:title=explainanalyze_5.q.out}
> 143c143
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> 154c154
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14900) fix entry for hive.exec.max.dynamic.partitions in config whitelist for sql std auth
[
https://issues.apache.org/jira/browse/HIVE-14900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-14900:
-
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Patch committed to master.
Thanks for the review Daniel!
> fix entry for hive.exec.max.dynamic.partitions in config whitelist for sql
> std auth
> ---
>
> Key: HIVE-14900
> URL: https://issues.apache.org/jira/browse/HIVE-14900
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 2.2.0
>
> Attachments: HIVE-14900.1.patch, HIVE-14900.1.patch,
> HIVE-14900.1.patch
>
>
> HiveConf.java has -
> {code}
> static final String [] sqlStdAuthSafeVarNameRegexes = new String [] {
> ...
> "hive\\.exec\\..*\\.dynamic\\.partitions\\..*",
> {code}
> The regex doesn't work for hive.exec.max.dynamic.partitions as there is a "."
> at the end.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15084) Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
[
https://issues.apache.org/jira/browse/HIVE-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785739#comment-15785739
]
Thejas M Nair commented on HIVE-15084:
--
[~pxiong]
Can you please take a look at this ? These tests have been flaky for a while
now.
> Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
>
>
> Key: HIVE-15084
> URL: https://issues.apache.org/jira/browse/HIVE-15084
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>
> Example diffs:
> {noformat:title=explainanalyze_2.q.out}
> 1881c1881
> < Group By Operator [GBY_2] (rows=1/500
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1
> > width=8)
> 2227c2227
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2237c2237
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> 2243c2243
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2260c2260
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> {noformat}
> {noformat:title=explainanalyze_2.q.out}
> 367c367
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 378c378
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 2135c2135
> < Group By Operator [GBY_2] (rows=1/241
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1 width=8)
> {noformat}
> From https://builds.apache.org/job/PreCommit-HIVE-Build/2295/testReport/:
> {noformat:title=explainanalyze_4.q.out}
> 248c248
> < Group By Operator [GBY_10] (rows=615/10 width=12)
> ---
> > Group By Operator [GBY_10] (rows=615/5 width=12)
> {noformat}
> {noformat:title=explainanalyze_5.q.out}
> 143c143
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> 154c154
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-15480) Failing test : TestMiniTezCliDriver.testCliDriver : explainanalyze_1
[
https://issues.apache.org/jira/browse/HIVE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair resolved HIVE-15480.
--
Resolution: Duplicate
> Failing test : TestMiniTezCliDriver.testCliDriver : explainanalyze_1
>
>
> Key: HIVE-15480
> URL: https://issues.apache.org/jira/browse/HIVE-15480
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sushanth Sowmyan
>
> See recent ptest failure :
> https://builds.apache.org/job/PreCommit-HIVE-Build/2642/testReport/org.apache.hadoop.hive.cli/TestMiniTezCliDriver/testCliDriver_explainanalyze_1_/
> {noformat}
> Standard Output
> Running: diff -a
> /home/hiveptest/104.154.92.121-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/explainanalyze_1.q.out
>
> /home/hiveptest/104.154.92.121-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/tez/explainanalyze_1.q.out
> 248c248
> < Group By Operator [GBY_2] (rows=205/500 width=95)
> ---
> > Group By Operator [GBY_2] (rows=205/309 width=95)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15084) Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
[
https://issues.apache.org/jira/browse/HIVE-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785736#comment-15785736
]
Thejas M Nair commented on HIVE-15084:
--
explainanalyze_1 as similar diff as well, as mentioned in HIVE-15480
> Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
>
>
> Key: HIVE-15084
> URL: https://issues.apache.org/jira/browse/HIVE-15084
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>
> Example diffs:
> {noformat:title=explainanalyze_2.q.out}
> 1881c1881
> < Group By Operator [GBY_2] (rows=1/500
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1
> > width=8)
> 2227c2227
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2237c2237
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> 2243c2243
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2260c2260
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> {noformat}
> {noformat:title=explainanalyze_2.q.out}
> 367c367
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 378c378
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 2135c2135
> < Group By Operator [GBY_2] (rows=1/241
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1 width=8)
> {noformat}
> From https://builds.apache.org/job/PreCommit-HIVE-Build/2295/testReport/:
> {noformat:title=explainanalyze_4.q.out}
> 248c248
> < Group By Operator [GBY_10] (rows=615/10 width=12)
> ---
> > Group By Operator [GBY_10] (rows=615/5 width=12)
> {noformat}
> {noformat:title=explainanalyze_5.q.out}
> 143c143
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> 154c154
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15480) Failing test : TestMiniTezCliDriver.testCliDriver : explainanalyze_1
[
https://issues.apache.org/jira/browse/HIVE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785734#comment-15785734
]
Thejas M Nair commented on HIVE-15480:
--
Lets track in HIVE-15084, which has other explainanalyze tests failing with
similar diff
> Failing test : TestMiniTezCliDriver.testCliDriver : explainanalyze_1
>
>
> Key: HIVE-15480
> URL: https://issues.apache.org/jira/browse/HIVE-15480
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sushanth Sowmyan
>
> See recent ptest failure :
> https://builds.apache.org/job/PreCommit-HIVE-Build/2642/testReport/org.apache.hadoop.hive.cli/TestMiniTezCliDriver/testCliDriver_explainanalyze_1_/
> {noformat}
> Standard Output
> Running: diff -a
> /home/hiveptest/104.154.92.121-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/explainanalyze_1.q.out
>
> /home/hiveptest/104.154.92.121-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/tez/explainanalyze_1.q.out
> 248c248
> < Group By Operator [GBY_2] (rows=205/500 width=95)
> ---
> > Group By Operator [GBY_2] (rows=205/309 width=95)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15084) Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
[
https://issues.apache.org/jira/browse/HIVE-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-15084:
-
Summary: Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
(was: Flaky test: TestMiniTezCliDriver:explainanalyze_2, 3, 4, 5)
> Flaky test: TestMiniTezCliDriver:explainanalyze_1,2, 3, 4, 5
>
>
> Key: HIVE-15084
> URL: https://issues.apache.org/jira/browse/HIVE-15084
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>
> Example diffs:
> {noformat:title=explainanalyze_2.q.out}
> 1881c1881
> < Group By Operator [GBY_2] (rows=1/500
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1
> > width=8)
> 2227c2227
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2237c2237
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> 2243c2243
> < Group By Operator [GBY_11] (rows=250/392 width=280)
> ---
> > Group By Operator [GBY_11] (rows=250/310 width=280)
> 2260c2260
> < Group By Operator [GBY_17] (rows=501/392 width=464)
> ---
> > Group By Operator [GBY_17] (rows=501/310 width=464)
> {noformat}
> {noformat:title=explainanalyze_2.q.out}
> 367c367
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 378c378
> < Group By Operator
> [GBY_78] (rows=262/334 width=178)
> ---
> > Group By Operator
> > [GBY_78] (rows=262/331 width=178)
> 2135c2135
> < Group By Operator [GBY_2] (rows=1/241
> width=8)
> ---
> > Group By Operator [GBY_2] (rows=1/1 width=8)
> {noformat}
> From https://builds.apache.org/job/PreCommit-HIVE-Build/2295/testReport/:
> {noformat:title=explainanalyze_4.q.out}
> 248c248
> < Group By Operator [GBY_10] (rows=615/10 width=12)
> ---
> > Group By Operator [GBY_10] (rows=615/5 width=12)
> {noformat}
> {noformat:title=explainanalyze_5.q.out}
> 143c143
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> 154c154
> < Group By Operator [GBY_9]
> (rows=262/522 width=178)
> ---
> > Group By Operator [GBY_9]
> > (rows=262/331 width=178)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15519) Hive Decimal Type column scale is returning as zero
[
https://issues.apache.org/jira/browse/HIVE-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharat Viswanadham updated HIVE-15519:
--
Description:
Hive decimal type column precision is returning as zero, even though column has
precision set.
Example: col67 decimal(18,2) scale is returning as zero for that column.
Tried with below program.
{code}
System.out.println("Opening connection");
Class.forName("org.apache.hive.jdbc.HiveDriver");
Connection con =
DriverManager.getConnection("jdbc:hive2://x.x.x.x:1/default");
DatabaseMetaData dbMeta = con.getMetaData();
ResultSet rs = dbMeta.getColumns(null, "DEFAULT", "data_type_test",null);
while (rs.next()) {
if (rs.getString("COLUMN_NAME").equalsIgnoreCase("col48") ||
rs.getString("COLUMN_NAME").equalsIgnoreCase("col67") ||
rs.getString("COLUMN_NAME").equalsIgnoreCase("col68") ||
rs.getString("COLUMN_NAME").equalsIgnoreCase("col122")){
System.out.println(rs.getString("COLUMN_NAME") + "\t" +
rs.getString("COLUMN_SIZE") + "\t" + rs.getInt("DECIMAL_DIGITS"));
}
}
rs.close();
con.close();
} catch (Exception e) {
e.printStackTrace();
;
}
{code}
Default fetch size is 50. if any column no is under 50 with decimal type,
precision is returning properly, when the column no is greater than 50, scale
is returning as zero.
was:
Hive decimal type column precision is returning as zero, even though column has
precision set.
Example: col67 decimal(18,2) precison is returning as zero for that column.
Tried with below program.
{code}
System.out.println("Opening connection");
Class.forName("org.apache.hive.jdbc.HiveDriver");
Connection con =
DriverManager.getConnection("jdbc:hive2://x.x.x.x:1/default");
DatabaseMetaData dbMeta = con.getMetaData();
ResultSet rs = dbMeta.getColumns(null, "DEFAULT", "data_type_test",null);
while (rs.next()) {
if (rs.getString("COLUMN_NAME").equalsIgnoreCase("col48") ||
rs.getString("COLUMN_NAME").equalsIgnoreCase("col67") ||
rs.getString("COLUMN_NAME").equalsIgnoreCase("col68") ||
rs.getString("COLUMN_NAME").equalsIgnoreCase("col122")){
System.out.println(rs.getString("COLUMN_NAME") + "\t" +
rs.getString("COLUMN_SIZE") + "\t" + rs.getInt("DECIMAL_DIGITS"));
}
}
rs.close();
con.close();
} catch (Exception e) {
e.printStackTrace();
;
}
{code}
Default fetch size is 50. if any column no is under 50 with decimal type,
precision is returning properly, when the column no is greater than 50,
precision is returning as zero.
> Hive Decimal Type column scale is returning as zero
> ---
>
> Key: HIVE-15519
> URL: https://issues.apache.org/jira/browse/HIVE-15519
> Project: Hive
> Issue Type: Bug
> Components: Hive, JDBC
>Reporter: Bharat Viswanadham
>Assignee: Rui Li
>Priority: Critical
> Attachments: HIVE-15519.1.patch, HIVE-15519.2.patch,
> data_type_test(1).txt
>
>
> Hive decimal type column precision is returning as zero, even though column
> has precision set.
> Example: col67 decimal(18,2) scale is returning as zero for that column.
> Tried with below program.
> {code}
>System.out.println("Opening connection");
> Class.forName("org.apache.hive.jdbc.HiveDriver");
>Connection con =
> DriverManager.getConnection("jdbc:hive2://x.x.x.x:1/default");
> DatabaseMetaData dbMeta = con.getMetaData();
>ResultSet rs = dbMeta.getColumns(null, "DEFAULT", "data_type_test",null);
> while (rs.next()) {
> if (rs.getString("COLUMN_NAME").equalsIgnoreCase("col48") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col67") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col68") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col122")){
> System.out.println(rs.getString("COLUMN_NAME") + "\t" +
> rs.getString("COLUMN_SIZE") + "\t" + rs.getInt("DECIMAL_DIGITS"));
> }
>}
>rs.close();
>con.close();
> } catch (Exception e) {
>e.printStackTrace();
>;
> }
> {code}
> Default fetch size is 50. if any column no is under 50 with decimal type,
> precision is returning properly, when the column no is greater than 50, scale
> is returning as zero.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15390) Orc reader unnecessarily reading stripe footers with hive.optimize.index.filter set to true
[ https://issues.apache.org/jira/browse/HIVE-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785716#comment-15785716 ] Thejas M Nair commented on HIVE-15390: -- cc [~prasanth_j] [~rajesh.balamohan] [~gopalv] > Orc reader unnecessarily reading stripe footers with > hive.optimize.index.filter set to true > --- > > Key: HIVE-15390 > URL: https://issues.apache.org/jira/browse/HIVE-15390 > Project: Hive > Issue Type: Bug > Components: ORC >Affects Versions: 1.2.1 >Reporter: Abhishek Somani >Assignee: Abhishek Somani > Attachments: HIVE-15390.1.patch, HIVE-15390.patch > > > In a split given to a task, the task's orc reader is unnecessarily reading > stripe footers for stripes that are not its responsibility to read. This is > happening with hive.optimize.index.filter set to true. > Assuming one split per task(no tez grouping considered), a task should not > need to read beyond the split's end offset. Even in some split computation > strategies where a split's end offset can be in the middle of a stripe, it > should not need to read more than one stripe beyond the split's end offset(to > fully read a stripe that started in it). However I see that some tasks make > unnecessary filesystem calls to read all the stripe footers in a file from > the split start offset till the end of the file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15518) Refactoring rows and range related classes to put the window type on Window
[
https://issues.apache.org/jira/browse/HIVE-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785667#comment-15785667
]
Hive QA commented on HIVE-15518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845070/HIVE-15518.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10839 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=139)
[skewjoinopt15.q,vector_coalesce.q,orc_ppd_decimal.q,cbo_rp_lineage2.q,insert_into_with_schema.q,join_emit_interval.q,load_dyn_part3.q,auto_sortmerge_join_14.q,vector_null_projection.q,vector_cast_constant.q,mapjoin2.q,bucket_map_join_tez2.q,correlationoptimizer4.q,schema_evol_orc_acidvec_part_update.q,vectorization_12.q,vector_number_compare_projection.q,orc_merge_incompat3.q,vector_leftsemi_mapjoin.q,update_all_non_partitioned.q,multi_column_in_single.q,schema_evol_orc_nonvec_table.q,cbo_rp_semijoin.q,tez_insert_overwrite_local_directory_1.q,schema_evol_text_vecrow_table.q,vector_count.q,auto_sortmerge_join_15.q,vector_if_expr.q,delete_whole_partition.q,vector_decimal_6.q,sample1.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=151)
[smb_mapjoin_15.q,insert_values_partitioned.q,selectDistinctStar.q,bucket4.q,vectorized_distinct_gby.q,vector_groupby_mapjoin.q,insert_values_dynamic_partitioned.q,vector_nvl.q,join_nullsafe.q,vectorized_mapjoin.q,schema_evol_orc_vec_part_all_primitive.q,vectorized_shufflejoin.q,tez_smb_1.q,cbo_union.q,tez_vector_dynpart_hashjoin_1.q,filter_join_breaktask2.q,table_access_keys_stats.q,vector_data_types.q,multiMapJoin2.q,filter_join_breaktask.q,schema_evol_orc_nonvec_part.q,alter_merge_2_orc.q,vectorization_3.q,union4.q,auto_sortmerge_join_8.q,stats_based_fetch_decision.q,vectorized_date_funcs.q,auto_sortmerge_join_10.q,vector_varchar_simple.q,vector_decimal_udf2.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ptf]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_ptf]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=92)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2746/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2746/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2746/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845070 - PreCommit-HIVE-Build
> Refactoring rows and range related classes to put the window type on Window
> ---
>
> Key: HIVE-15518
> URL: https://issues.apache.org/jira/browse/HIVE-15518
> Project: Hive
> Issue Type: Sub-task
> Components: PTF-Windowing
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15518.1.patch, HIVE-15518.2.patch,
> HIVE-15518.3.patch
>
>
> {noformat}
> /*
>* - A Window Frame that has only the /start/boundary, then it is
> interpreted as:
> BETWEEN AND CURRENT ROW
>* - A Window Specification with an Order Specification and no Window
>* Frame is interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
>* - A Window Specification with no Order and no Window Frame is
> interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
>*/
> {noformat}
> The comments in WindowSpec above doesn't really match what it's claimed to
> do. Correct the comment to reduce the confusion.
> Also currently the window type is specified on each BoundarySpec but makes
> sense to put the type (rows or range) for each window.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15016) Run tests with Hadoop 3.0.0-alpha1
[ https://issues.apache.org/jira/browse/HIVE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785633#comment-15785633 ] Steve Loughran commented on HIVE-15016: --- don't think Hadoop is making much use of codahale or committed to any specific JAR, if you need to update the one there > Run tests with Hadoop 3.0.0-alpha1 > -- > > Key: HIVE-15016 > URL: https://issues.apache.org/jira/browse/HIVE-15016 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: Hadoop3Upstream.patch > > > Hadoop 3.0.0-alpha1 was released back on Sep/16 to allow other components run > tests against this new version before GA. > We should start running tests with Hive to validate compatibility against > Hadoop 3.0. > NOTE: The patch used to test must not be committed to Hive until Hadoop 3.0 > GA is released. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15518) Refactoring rows and range related classes to put the window type on Window
[
https://issues.apache.org/jira/browse/HIVE-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-15518:
Attachment: HIVE-15518.3.patch
patch-3: fix additional test failures.
> Refactoring rows and range related classes to put the window type on Window
> ---
>
> Key: HIVE-15518
> URL: https://issues.apache.org/jira/browse/HIVE-15518
> Project: Hive
> Issue Type: Sub-task
> Components: PTF-Windowing
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15518.1.patch, HIVE-15518.2.patch,
> HIVE-15518.3.patch
>
>
> {noformat}
> /*
>* - A Window Frame that has only the /start/boundary, then it is
> interpreted as:
> BETWEEN AND CURRENT ROW
>* - A Window Specification with an Order Specification and no Window
>* Frame is interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
>* - A Window Specification with no Order and no Window Frame is
> interpreted as:
> ROW BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
>*/
> {noformat}
> The comments in WindowSpec above doesn't really match what it's claimed to
> do. Correct the comment to reduce the confusion.
> Also currently the window type is specified on each BoundarySpec but makes
> sense to put the type (rows or range) for each window.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15519) Hive Decimal Type column scale is returning as zero
[
https://issues.apache.org/jira/browse/HIVE-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15784921#comment-15784921
]
Hive QA commented on HIVE-15519:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845022/HIVE-15519.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10863 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=144)
[vectorized_rcfile_columnar.q,vector_elt.q,explainuser_1.q,multi_insert.q,tez_dml.q,vector_bround.q,schema_evol_orc_acid_table.q,vector_when_case_null.q,orc_ppd_schema_evol_1b.q,vector_join30.q,vectorization_11.q,cte_3.q,update_tmp_table.q,vector_decimal_cast.q,groupby_grouping_id2.q,vector_decimal_round.q,tez_smb_empty.q,orc_merge6.q,vector_char_mapjoin1.q,vector_decimal_trailing.q,cte_5.q,tez_union.q,vector_decimal_2.q,columnStatsUpdateForStatsOptimizer_1.q,vector_outer_join3.q,schema_evol_text_vec_part_all_complex.q,tez_dynpart_hashjoin_2.q,auto_sortmerge_join_12.q,offset_limit.q,tez_union_multiinsert.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=113)
[bucketmapjoin3.q,union_date.q,cbo_gby.q,auto_join31.q,auto_sortmerge_join_1.q,join_cond_pushdown_unqual1.q,ppd_outer_join3.q,bucket_map_join_spark3.q,union28.q,statsfs.q,escape_sortby1.q,leftsemijoin.q,groupby_multi_single_reducer3.q,union_remove_6.q,join29.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=92)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
(batchId=93)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2745/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2745/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2745/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845022 - PreCommit-HIVE-Build
> Hive Decimal Type column scale is returning as zero
> ---
>
> Key: HIVE-15519
> URL: https://issues.apache.org/jira/browse/HIVE-15519
> Project: Hive
> Issue Type: Bug
> Components: Hive, JDBC
>Reporter: Bharat Viswanadham
>Assignee: Rui Li
>Priority: Critical
> Attachments: HIVE-15519.1.patch, HIVE-15519.2.patch,
> data_type_test(1).txt
>
>
> Hive decimal type column precision is returning as zero, even though column
> has precision set.
> Example: col67 decimal(18,2) precison is returning as zero for that column.
> Tried with below program.
> {code}
>System.out.println("Opening connection");
> Class.forName("org.apache.hive.jdbc.HiveDriver");
>Connection con =
> DriverManager.getConnection("jdbc:hive2://x.x.x.x:1/default");
> DatabaseMetaData dbMeta = con.getMetaData();
>ResultSet rs = dbMeta.getColumns(null, "DEFAULT", "data_type_test",null);
> while (rs.next()) {
> if (rs.getString("COLUMN_NAME").equalsIgnoreCase("col48") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col67") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col68") ||
> rs.getString("COLUMN_NAME").equalsIgnoreCase("col122")){
> System.out.println(rs.getString("COLUMN_NAME") + "\t" +
> rs.getString("COLUMN_SIZE") + "\t" + rs.getInt("DECIMAL_DIGITS"));
> }
>}
>rs.close();
>con.close();
> } catch (Exception e) {
>e.printStackTrace();
>;
> }
> {code}
> Default fetch size is 50. if any column no is under 50 with decimal type,
> precision is returning properly, when the column no is greater than 50,
> precision is returning as zero.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14900) fix entry for hive.exec.max.dynamic.partitions in config whitelist for sql std auth
[
https://issues.apache.org/jira/browse/HIVE-14900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15784817#comment-15784817
]
Hive QA commented on HIVE-14900:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845020/HIVE-14900.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10899 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1]
(batchId=92)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2744/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2744/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2744/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845020 - PreCommit-HIVE-Build
> fix entry for hive.exec.max.dynamic.partitions in config whitelist for sql
> std auth
> ---
>
> Key: HIVE-14900
> URL: https://issues.apache.org/jira/browse/HIVE-14900
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-14900.1.patch, HIVE-14900.1.patch,
> HIVE-14900.1.patch
>
>
> HiveConf.java has -
> {code}
> static final String [] sqlStdAuthSafeVarNameRegexes = new String [] {
> ...
> "hive\\.exec\\..*\\.dynamic\\.partitions\\..*",
> {code}
> The regex doesn't work for hive.exec.max.dynamic.partitions as there is a "."
> at the end.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
