[jira] [Created] (HIVE-24365) SWO should not create complex and redundant filter expressions

2020-11-10 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24365:
---

 Summary: SWO should not create complex and redundant filter 
expressions
 Key: HIVE-24365
 URL: https://issues.apache.org/jira/browse/HIVE-24365
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


for q88 we have complex and mostly unreadable filter expressions; because 
before merging 2 branches the TS filterexpression is pushed into a FIL operator.





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24364) Fix flakiness of TestLlapExtClientWithCloudDeploymentConfigs

2020-11-10 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24364:
---

 Summary: Fix flakiness of 
TestLlapExtClientWithCloudDeploymentConfigs
 Key: HIVE-24364
 URL: https://issues.apache.org/jira/browse/HIVE-24364
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


it failed recently a few times in some PRs for me
falky check:
http://ci.hive.apache.org/job/hive-flaky-check/139/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24360) SharedWorkOptimizer may create incorrect plans with DPPUnion

2020-11-05 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24360:
---

 Summary: SharedWorkOptimizer may create incorrect plans with 
DPPUnion
 Key: HIVE-24360
 URL: https://issues.apache.org/jira/browse/HIVE-24360
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24357) Exchange SWO table/algorithm strategy

2020-11-04 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24357:
---

 Summary: Exchange SWO table/algorithm strategy
 Key: HIVE-24357
 URL: https://issues.apache.org/jira/browse/HIVE-24357
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich


SWO right now runs like: 
{code}
for every strategy s: for every table t: try s for t
{code}
this results in that an earlier startegy may create a more entangled operator 
tree behind - in case its able to merge for a less prioritized table

it would probably make more sense to do:

{code}
for every table t: for every strategy s: try s for t
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24355) Partition doesn't have hashCode/equals

2020-11-04 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24355:
---

 Summary: Partition doesn't have hashCode/equals
 Key: HIVE-24355
 URL: https://issues.apache.org/jira/browse/HIVE-24355
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


this might cause some issues - it also prevents the SWO from merging TS 
operators which have partitions in the "pruned list"



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24320) TestMiniLlapLocal sometimes hangs because of some derby issues

2020-10-28 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24320:
---

 Summary: TestMiniLlapLocal sometimes hangs because of some derby 
issues
 Key: HIVE-24320
 URL: https://issues.apache.org/jira/browse/HIVE-24320
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


code in question is a slightly modified version of branch-3

opening ticket to make notes about the investigation

{code}
"dcce5fec-2365-4697-8a8f-04a4dfa5d9f5 main" #1 prio=5 os_prio=0 
tid=0x7fd7c000a800 nid=0x1de23 waiting on condition [0x7fd7c4b7]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0xc61635f0> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1981)
at 
org.apache.derby.impl.services.cache.CacheEntry.waitUntilIdentityIsSet(Unknown 
Source)
at 
org.apache.derby.impl.services.cache.ConcurrentCache.getEntry(Unknown Source)
at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown 
Source)
at 
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown 
Source)
at 
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown 
Source)
at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown 
Source)
at 
org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown 
Source)
at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source)
at 
org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown 
Source)
at 
org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown 
Source)
at 
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndexMinion(Unknown
 Source)
at 
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(Unknown
 Source)
at 
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSubKeyConstraint(Unknown
 Source)
at 
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getConstraintDescriptorViaIndex(Unknown
 Source)
at 
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getConstraintDescriptorsScan(Unknown
 Source)
at 
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getConstraintDescriptors(Unknown
 Source)
- locked <0xc615c9a8> (a 
org.apache.derby.iapi.sql.dictionary.ConstraintDescriptorList)
at 
org.apache.derby.iapi.sql.dictionary.TableDescriptor.getAllRelevantConstraints(Unknown
 Source)
at 
org.apache.derby.impl.sql.compile.DMLModStatementNode.getAllRelevantConstraints(Unknown
 Source)
at 
org.apache.derby.impl.sql.compile.DMLModStatementNode.bindConstraints(Unknown 
Source)
at org.apache.derby.impl.sql.compile.DeleteNode.bindStatement(Unknown 
Source)
at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
at 
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
 Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
- locked <0xc4bb5fd0> (a 
org.apache.derby.impl.jdbc.EmbedConnection)
at 
org.apache.derby.impl.jdbc.EmbedStatement.executeBatchElement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeLargeBatch(Unknown 
Source)
- locked <0xc4bb5fd0> (a 
org.apache.derby.impl.jdbc.EmbedConnection)
at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(Unknown 
Source)
at 
com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)
at 
com.zaxxer.hikari.pool.HikariProxyStatement.executeBatch(HikariProxyStatement.java)
at 
org.apache.hadoop.hive.metastore.txn.TxnDbUtil.executeQueriesInBatch(TxnDbUtil.java:658)
at 
org.apache.hadoop.hive.metastore.txn.TxnHandler.updateCommitIdAndCleanUpMetadata(TxnHandler.java:1338)
at 
org.apache.hadoop.hive.metastore.txn.TxnHandler.commitTxn(TxnHandler.java:1236)
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.commit_txn(HiveMetaStore.java:8315)
at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler

[jira] [Created] (HIVE-24312) Use column stats to remove "x is not null" filter conditions if they are redundant

2020-10-26 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24312:
---

 Summary: Use column stats to remove "x is not null" filter 
conditions if they are redundant
 Key: HIVE-24312
 URL: https://issues.apache.org/jira/browse/HIVE-24312
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


with HIVE-24241 SharedWorkOptimizer could further merge branches for some 
queries (ex: 
[query32|https://github.com/apache/hive/blob/db895f374bf63b77b683574fdf678bfac91a5ac6/ql/src/test/results/clientpositive/perf/tez/query32.q.out#L118-L163]
 )

...but a little `is not null` difference prevents it from proceeding.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24295) Apply schema merge to all shared work optimizations

2020-10-22 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24295:
---

 Summary: Apply schema merge to all shared work optimizations
 Key: HIVE-24295
 URL: https://issues.apache.org/jira/browse/HIVE-24295
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24269) In SharedWorkOptimizer run simplification after merging TS filter expressions

2020-10-13 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24269:
---

 Summary: In SharedWorkOptimizer run simplification after merging 
TS filter expressions
 Key: HIVE-24269
 URL: https://issues.apache.org/jira/browse/HIVE-24269
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24268) Investigate srcpart scans in dynamic_partition_pruning test

2020-10-13 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24268:
---

 Summary: Investigate srcpart scans in dynamic_partition_pruning 
test
 Key: HIVE-24268
 URL: https://issues.apache.org/jira/browse/HIVE-24268
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


there seems to be some opportunities missed by shared work optimizer

see srcpart scans around 
[here|https://github.com/apache/hive/pull/1553/files/31ddb97bf412a2679489a5a0d45d335d2708005c#diff-cb6322e933f130f318d462f2c3af839dac60f8acca2074b2685f1847066e3565R4313]





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24264) Fix failed-to-read errors in precommit runs

2020-10-12 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24264:
---

 Summary: Fix failed-to-read errors in precommit runs
 Key: HIVE-24264
 URL: https://issues.apache.org/jira/browse/HIVE-24264
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


the following happens:
* this seems to be caused by tests outputting a lot of messages
* some error happens in surefire - and the system-err is discarded
* junit xml becomes corrupted
* jenkins does report the failure - but doesn't take it into account in build 
result setting; so the result will remain green





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24265) Fix acid_stats2 test

2020-10-12 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24265:
---

 Summary: Fix acid_stats2 test
 Key: HIVE-24265
 URL: https://issues.apache.org/jira/browse/HIVE-24265
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


This test's failure started to create incorrect junit xmls which was not 
counted correctly by jenkins.

I'll disable the test now - and provide details on when it failed first.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24242) Relax safety checks in SharedWorkOptimizer

2020-10-08 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24242:
---

 Summary: Relax safety checks in SharedWorkOptimizer
 Key: HIVE-24242
 URL: https://issues.apache.org/jira/browse/HIVE-24242
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


there are some checks to lock out problematic cases

For UnionOperator 
[here|https://github.com/apache/hive/blob/1507d80fd47aad38b87bba4fd58c1427ba89dbbf/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1571])

This check could prevent the optimization even if the Union is only visible 
from only 1 of the TS ops.





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24241) Enable SharedWorkOptimizer to merge downstream operators after an optimization step

2020-10-08 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24241:
---

 Summary: Enable SharedWorkOptimizer to merge downstream operators 
after an optimization step
 Key: HIVE-24241
 URL: https://issues.apache.org/jira/browse/HIVE-24241
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24231) Enhance shared work optimizer to merge scans with semijoin filters on both sides

2020-10-05 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24231:
---

 Summary: Enhance shared work optimizer to merge scans with 
semijoin filters on both sides
 Key: HIVE-24231
 URL: https://issues.apache.org/jira/browse/HIVE-24231
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24172) Fix TestMmCompactorOnMr

2020-09-16 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24172:
---

 Summary: Fix TestMmCompactorOnMr
 Key: HIVE-24172
 URL: https://issues.apache.org/jira/browse/HIVE-24172
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


test is unstable;
http://ci.hive.apache.org/job/hive-flaky-check/112/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24160) Scheduled executions must allow state transitions to TIMED_OUT from any state

2020-09-14 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24160:
---

 Summary: Scheduled executions must allow state transitions to 
TIMED_OUT from any state
 Key: HIVE-24160
 URL: https://issues.apache.org/jira/browse/HIVE-24160
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24130) Support datasets for non-default database

2020-09-08 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24130:
---

 Summary: Support datasets for non-default database
 Key: HIVE-24130
 URL: https://issues.apache.org/jira/browse/HIVE-24130
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


tpch datasets were added in a different database - but QTestDatasetHandler only 
considers tables by "name" and ignores the "db" - so the protection mechanism 
doesn't fully work for the tpch tables are being wiped out after the first test 
using them and never loaded back



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24123) Improve cost model for Aggregates

2020-09-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24123:
---

 Summary: Improve cost model for Aggregates
 Key: HIVE-24123
 URL: https://issues.apache.org/jira/browse/HIVE-24123
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24084) Enhance cost model to push down more Aggregates

2020-08-27 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24084:
---

 Summary: Enhance cost model to push down more Aggregates
 Key: HIVE-24084
 URL: https://issues.apache.org/jira/browse/HIVE-24084
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24072) HiveAggregateJoinTransposeRule may try to create an invalid transformation

2020-08-25 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-24072:
---

 Summary: HiveAggregateJoinTransposeRule may try to create an 
invalid transformation
 Key: HIVE-24072
 URL: https://issues.apache.org/jira/browse/HIVE-24072
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


{code}
java.lang.AssertionError: 
Cannot add expression of different type to set:
set type is RecordType(INTEGER NOT NULL o_orderkey, DECIMAL(10, 0) 
o_totalprice, DATE o_orderdate, INTEGER NOT NULL c_custkey, VARCHAR(25) 
CHARACTER SET "UTF-16LE" c_name, DOUBLE $f5) NOT NULL
expression type is RecordType(INTEGER NOT NULL o_orderkey, INTEGER NOT NULL 
o_custkey, DECIMAL(10, 0) o_totalprice, DATE o_orderdate, INTEGER NOT NULL 
c_custkey, DOUBLE $f1) NOT NULL
set is rel#567:HiveAggregate.HIVE.[].any(input=HepRelVertex#490,group={2, 4, 5, 
6, 7},agg#0=sum($1))
expression is HiveProject(o_orderkey=[$2], o_custkey=[$3], o_totalprice=[$4], 
o_orderdate=[$5], c_custkey=[$6], $f1=[$1])
  HiveJoin(condition=[=($2, $0)], joinType=[inner], algorithm=[none], 
cost=[{2284.5 rows, 0.0 cpu, 0.0 io}])
HiveAggregate(group=[{0}], agg#0=[sum($1)])
  HiveProject(l_orderkey=[$0], l_quantity=[$4])
HiveTableScan(table=[[tpch_0_001, lineitem]], table:alias=[l])
HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], 
cost=[{1.9115E15 rows, 0.0 cpu, 0.0 io}])
  HiveJoin(condition=[=($4, $1)], joinType=[inner], algorithm=[none], 
cost=[{1650.0 rows, 0.0 cpu, 0.0 io}])
HiveProject(o_orderkey=[$0], o_custkey=[$1], o_totalprice=[$3], 
o_orderdate=[$4])
  HiveTableScan(table=[[tpch_0_001, orders]], table:alias=[orders])
HiveProject(c_custkey=[$0], c_name=[$1])
  HiveTableScan(table=[[tpch_0_001, customer]], table:alias=[customer])
  HiveProject($f0=[$0])
HiveFilter(condition=[>($1, 3E2)])
  HiveAggregate(group=[{0}], agg#0=[sum($4)])
HiveTableScan(table=[[tpch_0_001, lineitem]], 
table:alias=[lineitem])

at 
org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:383)
at 
org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
at 
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:236)
at 
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveAggregateJoinTransposeRule.onMatch(HiveAggregateJoinTransposeRule.java:300)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23986) flaky TestStatsReplicationScenariosMigration.testMetadataOnlyDump

2020-08-04 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23986:
---

 Summary: flaky 
TestStatsReplicationScenariosMigration.testMetadataOnlyDump
 Key: HIVE-23986
 URL: https://issues.apache.org/jira/browse/HIVE-23986
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/143/testReport/junit/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23985) flaky TestMiniHiveKafkaCliDriver.testCliDriver[kafka_storage_handler]

2020-08-04 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23985:
---

 Summary: flaky 
TestMiniHiveKafkaCliDriver.testCliDriver[kafka_storage_handler]
 Key: HIVE-23985
 URL: https://issues.apache.org/jira/browse/HIVE-23985
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/144/testReport/junit/org.apache.hadoop.hive.cli/TestMiniHiveKafkaCliDriver/Testing___split_16___Archive___testCliDriver_kafka_storage_handler_/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23984) flaky TestMiniDruidCliDriver

2020-08-04 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23984:
---

 Summary: flaky TestMiniDruidCliDriver 
 Key: HIVE-23984
 URL: https://issues.apache.org/jira/browse/HIVE-23984
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/146/testReport/junit/org.apache.hadoop.hive.cli/TestMiniDruidCliDriver/Testing___split_13___Archive___testCliDriver_druidmini_expressions_/

http://ci.hive.apache.org/job/hive-precommit/job/master/146/testReport/junit/org.apache.hadoop.hive.cli/TestMiniDruidCliDriver/Testing___split_13___Archive___testCliDriver_druidmini_dynamic_partition_/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23983) TestConcurrentJobRequestsThreadsAndTimeout is flaky

2020-08-04 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23983:
---

 Summary: TestConcurrentJobRequestsThreadsAndTimeout is flaky
 Key: HIVE-23983
 URL: https://issues.apache.org/jira/browse/HIVE-23983
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/151/testReport/junit/org.apache.hive.hcatalog.templeton/TestConcurrentJobRequestsThreadsAndTimeout/Testing___split_04___Archive___ConcurrentListJobsVerifyExceptions/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23982) TestStatsReplicationScenariosMigrationNoAutogather is flaky

2020-08-04 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23982:
---

 Summary: TestStatsReplicationScenariosMigrationNoAutogather is 
flaky
 Key: HIVE-23982
 URL: https://issues.apache.org/jira/browse/HIVE-23982
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/148/testReport/junit/org.apache.hadoop.hive.ql.parse/TestStatsReplicationScenariosMigrationNoAutogather/Testing___split_16___Archive___testRetryFailure/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23959) Provide an option to wipe out column stats for partitioned tables in case of column removal

2020-07-30 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23959:
---

 Summary: Provide an option to wipe out column stats for 
partitioned tables in case of column removal
 Key: HIVE-23959
 URL: https://issues.apache.org/jira/browse/HIVE-23959
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


in case of column removal / replacement - an update for each partition is 
neccessary; which could take a while.

goal here is to provide an option to switch to the bulk removal of column 
statistics instead of working hard to retain as much as possible from the old 
stats.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23945) Fix TestCodahaleReportersConf

2020-07-29 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23945:
---

 Summary: Fix TestCodahaleReportersConf
 Key: HIVE-23945
 URL: https://issues.apache.org/jira/browse/HIVE-23945
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/138/testReport/junit/org.apache.hadoop.hive.common.metrics.metrics2/TestCodahaleReportersConf/Testing___split_11___Archive___testNoFallback/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23944) Stabilize TestStatsReplicationScenariosMMNoAutogather

2020-07-29 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23944:
---

 Summary: Stabilize TestStatsReplicationScenariosMMNoAutogather
 Key: HIVE-23944
 URL: https://issues.apache.org/jira/browse/HIVE-23944
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/130/testReport/junit/org.apache.hadoop.hive.ql.parse/TestStatsReplicationScenariosMMNoAutogather/Testing___split_16___Archive___testMetadataOnlyDump/
http://ci.hive.apache.org/job/hive-precommit/job/PR-1303/lastCompletedBuild/testReport/org.apache.hadoop.hive.ql.parse/TestStatsReplicationScenariosMigrationNoAutogather/Testing___split_09___Archive___testRetryFailure/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23926) Flaky test TestTableLevelReplicationScenarios.testRenameTableScenariosWithReplacePolicyDMLOperattion

2020-07-24 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23926:
---

 Summary: Flaky test 
TestTableLevelReplicationScenarios.testRenameTableScenariosWithReplacePolicyDMLOperattion
 Key: HIVE-23926
 URL: https://issues.apache.org/jira/browse/HIVE-23926
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/123/testReport/org.apache.hadoop.hive.ql.parse/TestTableLevelReplicationScenarios/Testing___split_18___Archive___testRenameTableScenariosWithReplacePolicyDMLOperattion/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23925) Flaky test TestJdbcWithMiniHS2

2020-07-24 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23925:
---

 Summary: Flaky test TestJdbcWithMiniHS2
 Key: HIVE-23925
 URL: https://issues.apache.org/jira/browse/HIVE-23925
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/122/testReport/org.apache.hive.jdbc/TestJdbcWithMiniHS2/Testing___split_10___Archive___testConcurrentStatements/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23924) Flaky test TestStatsReplicationScenariosACID.testForParallelBootstrapLoad

2020-07-24 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23924:
---

 Summary: Flaky test 
TestStatsReplicationScenariosACID.testForParallelBootstrapLoad
 Key: HIVE-23924
 URL: https://issues.apache.org/jira/browse/HIVE-23924
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-precommit/job/master/122/testReport/org.apache.hadoop.hive.ql.parse/TestStatsReplicationScenariosACID/Testing___split_08___Archive___testForParallelBootstrapLoad/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23923) Stabilize resultset of schema_evol_text_vec_part_all_primitive

2020-07-24 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23923:
---

 Summary: Stabilize resultset of 
schema_evol_text_vec_part_all_primitive
 Key: HIVE-23923
 URL: https://issues.apache.org/jira/browse/HIVE-23923
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


http://ci.hive.apache.org/job/hive-flaky-check/80/testReport/junit/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_schema_evol_text_vec_part_all_primitive_/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23910) Some jdbc tests are unstable

2020-07-23 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23910:
---

 Summary: Some jdbc tests are unstable
 Key: HIVE-23910
 URL: https://issues.apache.org/jira/browse/HIVE-23910
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


they fail from time-to-time because of "no such scheme" or "no such table" or 
similar...

http://ci.hive.apache.org/job/hive-precommit/job/master/116/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23855) TestQueryShutdownHooks is flaky

2020-07-15 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23855:
---

 Summary: TestQueryShutdownHooks is flaky
 Key: HIVE-23855
 URL: https://issues.apache.org/jira/browse/HIVE-23855
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich



http://ci.hive.apache.org/job/hive-precommit/job/master/100/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23828) Consider supporting LONG as a column type

2020-07-09 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23828:
---

 Summary: Consider supporting LONG as a column type
 Key: HIVE-23828
 URL: https://issues.apache.org/jira/browse/HIVE-23828
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich


right now there seems to be trace that LONG was once supported (KW_LONG in 
parser)
and in statsutils

https://github.com/apache/hive/pull/1229#discussion_r452267303



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23827) Upgrade to datasketches 1.1.0

2020-07-09 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23827:
---

 Summary: Upgrade to datasketches 1.1.0
 Key: HIVE-23827
 URL: https://issues.apache.org/jira/browse/HIVE-23827
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23806) Avoid clearing column stat states in all partition in case schema is extended

2020-07-06 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23806:
---

 Summary: Avoid clearing column stat states in all partition in 
case schema is extended
 Key: HIVE-23806
 URL: https://issues.apache.org/jira/browse/HIVE-23806
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23776) Retire quickstats autocollection

2020-06-29 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23776:
---

 Summary: Retire quickstats autocollection
 Key: HIVE-23776
 URL: https://issues.apache.org/jira/browse/HIVE-23776
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


this is about:
* num files
* datasize (sum of filesizes)
* num erasure coded files

right now these are scanned during every BasicStatsTask execution - which means 
some filesystem reads/etc - for small inserts these are visible in case the fs 
is a bit slower (s3 and friends)

I don't think they are really in use...we rely more on columnstats which are 
more accurate ; and because of the datasize in this case is for "offline" 
(ondisk) - while we should be insted calculate with "online" sizes...

proposal:

* remove collection and storage of this data
* collect it on the fly during "desc formatted" statements to provide them for 
informational purposes



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23775) investigate windowing spec when an order by is present

2020-06-29 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23775:
---

 Summary: investigate windowing spec when an order by is present
 Key: HIVE-23775
 URL: https://issues.apache.org/jira/browse/HIVE-23775
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


this is some weird stuff came up during review

https://github.com/apache/hive/pull/1126#discussion_r442266978

Order by spec -> range, unbounded preceding, current row
This also aligns with most RDBMSs implementation





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23728) Run metastore verification tests during precommit

2020-06-19 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23728:
---

 Summary: Run metastore verification tests during precommit
 Key: HIVE-23728
 URL: https://issues.apache.org/jira/browse/HIVE-23728
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23713) Move shaded perf builds into a separate profile

2020-06-17 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23713:
---

 Summary: Move shaded perf builds into a separate profile
 Key: HIVE-23713
 URL: https://issues.apache.org/jira/browse/HIVE-23713
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


they are sometimes even use the "dist" profile

not building these things makes things a bit faster during testing
right now (including the sources; built artifacts - and the dependent maven 
stuff as well):

// 2.9G w/o dist
// 4.9G w/ dist
// 3.9G w/ perf





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23709) jdbc_handler is flaky

2020-06-17 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23709:
---

 Summary: jdbc_handler is flaky
 Key: HIVE-23709
 URL: https://issues.apache.org/jira/browse/HIVE-23709
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich



http://34.66.156.144:8080/job/hive-precommit/job/master/51/testReport/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23694) TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic] resultset unstable

2020-06-15 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23694:
---

 Summary: 
TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic] resultset 
unstable
 Key: HIVE-23694
 URL: https://issues.apache.org/jira/browse/HIVE-23694
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich




http://130.211.9.232/job/hive-precommit/job/master/45/testReport/junit/org.apache.hadoop.hive.cli/TestMiniDruidKafkaCliDriver/Testing___split_19___Archive___testCliDriver_druidkafkamini_basic_/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23693) HiveKafkaProducerTest.resumeTransaction is flaky

2020-06-15 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23693:
---

 Summary: HiveKafkaProducerTest.resumeTransaction is flaky
 Key: HIVE-23693
 URL: https://issues.apache.org/jira/browse/HIVE-23693
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


http://130.211.9.232/job/hive-precommit/job/master/44/testReport/junit/org.apache.hadoop.hive.kafka/HiveKafkaProducerTest/Testing___split_10___Archive___resumeTransaction/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23692) TestCodahaleMetrics.testFileReporting is flaky

2020-06-15 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23692:
---

 Summary: TestCodahaleMetrics.testFileReporting is flaky
 Key: HIVE-23692
 URL: https://issues.apache.org/jira/browse/HIVE-23692
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


http://130.211.9.232/job/hive-precommit/job/master/43/testReport/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23691) TestMiniLlapLocalCliDriver#testCliDriver[schq_materialized] is flaky

2020-06-15 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23691:
---

 Summary: 
TestMiniLlapLocalCliDriver#testCliDriver[schq_materialized] is flaky
 Key: HIVE-23691
 URL: https://issues.apache.org/jira/browse/HIVE-23691
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich



http://34.66.156.144:8080/job/hive-precommit/job/master/39/testReport/junit/org.apache.hadoop.hive.cli.split20/TestMiniLlapLocalCliDriver/Testing___split_13___Archive___testCliDriver_schq_materialized_/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23690) TestNegativeCliDriver#[external_jdbc_negative] is flaky

2020-06-15 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23690:
---

 Summary: TestNegativeCliDriver#[external_jdbc_negative] is flaky
 Key: HIVE-23690
 URL: https://issues.apache.org/jira/browse/HIVE-23690
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


failed after 10 tries:
http://130.211.9.232/job/hive-flaky-check/34/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23682) TestMetrics is flaky

2020-06-12 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23682:
---

 Summary: TestMetrics is flaky
 Key: HIVE-23682
 URL: https://issues.apache.org/jira/browse/HIVE-23682
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


http://34.66.156.144:8080/job/hive-precommit/job/master/31/testReport/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23681) TestTriggersMoveWorkloadManager is unstable

2020-06-12 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23681:
---

 Summary: TestTriggersMoveWorkloadManager is unstable
 Key: HIVE-23681
 URL: https://issues.apache.org/jira/browse/HIVE-23681
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


http://34.66.156.144:8080/job/hive-precommit/job/master/37/testReport/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23680) TestDbNotificationListener is unstable

2020-06-12 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23680:
---

 Summary: TestDbNotificationListener is unstable
 Key: HIVE-23680
 URL: https://issues.apache.org/jira/browse/HIVE-23680
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


http://34.66.156.144:8080/job/hive-precommit/job/master/35/testReport/
http://130.211.9.232/job/hive-flaky-check/24/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23679) TestSparkClient is flaky

2020-06-12 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23679:
---

 Summary: TestSparkClient is flaky
 Key: HIVE-23679
 URL: https://issues.apache.org/jira/browse/HIVE-23679
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


http://130.211.9.232/job/hive-precommit/job/master/34/testReport/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23677) RetryTest is unstable

2020-06-11 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23677:
---

 Summary: RetryTest is unstable
 Key: HIVE-23677
 URL: https://issues.apache.org/jira/browse/HIVE-23677
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


it failed on master

http://34.66.156.144:8080/job/hive-precommit/job/master/33/testReport/org.apache.hadoop.hive.metastore.utils/RetryTest/Testing___split_08___Archive___testRetryFailureWithDelayMoreThanTimeout/


and it took only 2 runs to make it fail using the flaky checker:
http://34.66.156.144:8080/job/hive-flaky-check/20/




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23669) Stop using jetty-runner which is an uberjar

2020-06-10 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23669:
---

 Summary: Stop using jetty-runner which is an uberjar
 Key: HIVE-23669
 URL: https://issues.apache.org/jira/browse/HIVE-23669
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


we already create our own shaded artifacts ; during it detects class 
duplications
{code}
02:51:03  [WARNING] jetty-server-9.3.27.v20190418.jar, 
jetty-runner-9.3.27.v20190418.jar define 245 overlapping classes: 
02:51:03  [WARNING]   - 
org.eclipse.jetty.server.handler.ThreadLimitHandler$RFC7239
02:51:03  [WARNING]   - 
org.eclipse.jetty.server.handler.ManagedAttributeListener
02:51:03  [WARNING]   - 
org.eclipse.jetty.server.session.JDBCSessionIdManager$SessionTableSchema
02:51:03  [WARNING]   - org.eclipse.jetty.server.ServletRequestHttpWrapper
02:51:03  [WARNING]   - org.eclipse.jetty.server.DebugListener$1
02:51:03  [WARNING]   - 
org.eclipse.jetty.server.session.HashSessionManager$Saver
02:51:03  [WARNING]   - org.eclipse.jetty.server.ShutdownMonitor$1
02:51:03  [WARNING]   - org.eclipse.jetty.server.HttpInput$1
02:51:03  [WARNING]   - org.eclipse.jetty.server.UserIdentity$Scope
02:51:03  [WARNING]   - org.eclipse.jetty.server.AsyncContextState$1
02:51:03  [WARNING]   - 235 more...
02:51:03  [WARNING] jetty-runner-9.3.27.v20190418.jar, ecj-4.4.2.jar define 557 
overlapping classes: 
02:51:03  [WARNING]   - 
org.eclipse.jdt.internal.compiler.apt.model.ErrorTypeElement
02:51:03  [WARNING]   - 
org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding
02:51:03  [WARNING]   - 
org.eclipse.jdt.internal.compiler.lookup.ConstraintFormula
02:51:03  [WARNING]   - 
org.eclipse.jdt.internal.compiler.apt.dispatch.BatchMessagerImpl
02:51:03  [WARNING]   - org.eclipse.jdt.internal.compiler.ast.MarkerAnnotation
02:51:03  [WARNING]   - org.eclipse.jdt.internal.compiler.ast.ThrowStatement
02:51:03  [WARNING]   - 
org.eclipse.jdt.internal.compiler.ast.JavadocSingleNameReference
02:51:03  [WARNING]   - 
org.eclipse.jdt.internal.compiler.lookup.ProblemReferenceBinding
02:51:03  [WARNING]   - org.eclipse.jdt.internal.compiler.ast.IntLiteral
02:51:03  [WARNING]   - org.eclipse.jdt.internal.compiler.ast.Reference
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23662) Run a smoketest during precommit

2020-06-08 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23662:
---

 Summary: Run a smoketest during precommit
 Key: HIVE-23662
 URL: https://issues.apache.org/jira/browse/HIVE-23662
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23661) Run metastore integration tests during precommit

2020-06-08 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23661:
---

 Summary: Run metastore integration tests during precommit
 Key: HIVE-23661
 URL: https://issues.apache.org/jira/browse/HIVE-23661
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23660) Provide a way to check test stability

2020-06-08 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23660:
---

 Summary: Provide a way to check test stability
 Key: HIVE-23660
 URL: https://issues.apache.org/jira/browse/HIVE-23660
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23631) Use the test target instead of install

2020-06-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23631:
---

 Summary: Use the test target instead of install
 Key: HIVE-23631
 URL: https://issues.apache.org/jira/browse/HIVE-23631
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23630) TestTxnHandler#allocateNextWriteIdRetriesAfterDetectingConflictingConcurrentInsert is unstable

2020-06-06 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23630:
---

 Summary: 
TestTxnHandler#allocateNextWriteIdRetriesAfterDetectingConflictingConcurrentInsert
 is unstable
 Key: HIVE-23630
 URL: https://issues.apache.org/jira/browse/HIVE-23630
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


I think I've seen this exception but it seems to be coming from this testcase 
more frequently
{code}
java.lang.RuntimeException: MetaException(message:Unable to update transaction 
database 
org.apache.derby.shared.common.error.DerbySQLIntegrityConstraintViolationException:
 The statement was aborted because it would have caused a duplicate key value 
in a unique or primary key constraint or unique index identified by 
'NEXT_WRITE_ID_IDX' defined on 'NEXT_WRITE_ID'.
 at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown 
Source)
 at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
 at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
Source)
 at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown 
Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
 at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown 
Source)
 at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
 at 
com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
 at 
com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
 at 
org.apache.hadoop.hive.metastore.txn.TxnHandler.allocateTableWriteIds(TxnHandler.java:2074)
 at 
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.lambda$allocateNextWriteIdRetriesAfterDetectingConflictingConcurrentInsert$1(TestTxnHandler.java:1780)
 at 
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
 at 
java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
 at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
 at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
 at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
 at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: ERROR 23505: The statement was aborted because it would have caused 
a duplicate key value in a unique or primary key constraint or unique index 
identified by 'NEXT_WRITE_ID_IDX' defined on 'NEXT_WRITE_ID'.
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown 
Source)
 at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
 at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
 at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
 at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
 at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown 
Source)
 at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
 at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown 
Source)
 at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
 ... 13 more
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23626) Build failure is incorrectly reported as tests passed

2020-06-05 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23626:
---

 Summary: Build failure is incorrectly reported as tests passed
 Key: HIVE-23626
 URL: https://issues.apache.org/jira/browse/HIVE-23626
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23622) Fix orc_merge9

2020-06-05 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23622:
---

 Summary: Fix orc_merge9 
 Key: HIVE-23622
 URL: https://issues.apache.org/jira/browse/HIVE-23622
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23621) Enforce ASF headers on source files

2020-06-05 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23621:
---

 Summary: Enforce ASF headers on source files
 Key: HIVE-23621
 URL: https://issues.apache.org/jira/browse/HIVE-23621
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23600) Enable users to provide the construction parameters for sketches

2020-06-03 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23600:
---

 Summary: Enable users to provide the construction parameters for 
sketches
 Key: HIVE-23600
 URL: https://issues.apache.org/jira/browse/HIVE-23600
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23599) CUME_DIST should not return NULL when partitioning column is NULL

2020-06-03 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23599:
---

 Summary: CUME_DIST should not return NULL when partitioning column 
is NULL
 Key: HIVE-23599
 URL: https://issues.apache.org/jira/browse/HIVE-23599
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


exposed by:
ql/src/test/results/clientpositive/llap/sketches_rewrite_cume_dist_partition_by.q.out

postgres also doesn't return NULLs in these cases

{code}
SELECT id,category, CUME_DIST() OVER (partition by category ORDER BY id) FROM 
sketch_input order by category,id;
{code}





--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23598) Add option to rewrite NTILE to sketch functions

2020-06-03 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23598:
---

 Summary: Add option to rewrite NTILE to sketch functions
 Key: HIVE-23598
 URL: https://issues.apache.org/jira/browse/HIVE-23598
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23594) Map more windowing aggregate functions to calcite

2020-06-02 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23594:
---

 Summary: Map more windowing aggregate functions to calcite
 Key: HIVE-23594
 URL: https://issues.apache.org/jira/browse/HIVE-23594
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


we have a few methods which are not mapped like CUME_DIST ; there might be a 
few more



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23593) Schemainit fails with NoSuchFieldError

2020-06-02 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23593:
---

 Summary: Schemainit fails with NoSuchFieldError 
 Key: HIVE-23593
 URL: https://issues.apache.org/jira/browse/HIVE-23593
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


the issue comes from a calcite related class ; it's very interesting because ql 
already has a shaded calcite
{code}
Caused by: java.lang.NoSuchFieldError: operands
at 
org.apache.hadoop.hive.ql.optimizer.calcite.translator.ExprNodeConverter.visitCall(ExprNodeConverter.java:192)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.optimizer.calcite.translator.ExprNodeConverter.visitCall(ExprNodeConverter.java:98)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) 
~[calcite-core-1.21.0.jar:1.21.0]
at 
org.apache.hadoop.hive.ql.optimizer.calcite.HiveRexExecutorImpl.reduce(HiveRexExecutorImpl.java:56)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.type.HiveFunctionHelper.foldExpression(HiveFunctionHelper.java:544)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.type.HiveFunctionHelper.createConstantObjectInspector(HiveFunctionHelper.java:452)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.type.HiveFunctionHelper.createObjectInspector(HiveFunctionHelper.java:435)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.type.HiveFunctionHelper.getReturnType(HiveFunctionHelper.java:124)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.type.RexNodeExprFactory.createFuncCallExpr(RexNodeExprFactory.java:647)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23590) Close stale PRs automatically

2020-06-02 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23590:
---

 Summary: Close stale PRs automatically
 Key: HIVE-23590
 URL: https://issues.apache.org/jira/browse/HIVE-23590
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23579) Introduce ReturnTypeInference for sketch functions

2020-05-29 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23579:
---

 Summary: Introduce ReturnTypeInference for sketch functions
 Key: HIVE-23579
 URL: https://issues.apache.org/jira/browse/HIVE-23579
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


Currently all sketch function's return type is processed thru the UDF api - 
which is not the best.

A better approach would be to somehow tie in the GenericUDF#initialize method



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23578) Collect ignored tests

2020-05-29 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23578:
---

 Summary: Collect ignored tests
 Key: HIVE-23578
 URL: https://issues.apache.org/jira/browse/HIVE-23578
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23568) special_character_in_tabnames_1.q is unstable

2020-05-28 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23568:
---

 Summary: special_character_in_tabnames_1.q is unstable
 Key: HIVE-23568
 URL: https://issues.apache.org/jira/browse/HIVE-23568
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23567) authorization_disallow_transform.q is unstable

2020-05-28 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23567:
---

 Summary: authorization_disallow_transform.q is unstable
 Key: HIVE-23567
 URL: https://issues.apache.org/jira/browse/HIVE-23567
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23565) temp_table_multi_insert_partitioned.q is unstable

2020-05-28 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23565:
---

 Summary: temp_table_multi_insert_partitioned.q is unstable
 Key: HIVE-23565
 URL: https://issues.apache.org/jira/browse/HIVE-23565
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23566) schema_evol_par_vec_table_* tests are unstable

2020-05-28 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23566:
---

 Summary: schema_evol_par_vec_table_* tests are unstable
 Key: HIVE-23566
 URL: https://issues.apache.org/jira/browse/HIVE-23566
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23564) TestSlotZnode concurrency tests are unstable

2020-05-28 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23564:
---

 Summary: TestSlotZnode concurrency tests are unstable
 Key: HIVE-23564
 URL: https://issues.apache.org/jira/browse/HIVE-23564
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23563) Evaluate options to do an early abort of the build

2020-05-28 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23563:
---

 Summary: Evaluate options to do an early abort of the build
 Key: HIVE-23563
 URL: https://issues.apache.org/jira/browse/HIVE-23563
 Project: Hive
  Issue Type: Sub-task
  Components: Build Infrastructure
Reporter: Zoltan Haindrich


if the job is waiting for a long to acquire the lock for a long time; it would 
be favourable to do a quick check after lock acqusition wether there are new 
triggers

https://github.com/apache/hive/pull/948#discussion_r432093713



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23557) HiveVolcanoPlanner works with 2 different RelMetadataQuery types which may lead to problems

2020-05-27 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23557:
---

 Summary: HiveVolcanoPlanner works with 2 different 
RelMetadataQuery types which may lead to problems
 Key: HIVE-23557
 URL: https://issues.apache.org/jira/browse/HIVE-23557
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


switching to trace mode uncovers Volcano estimation issues; which lead to the 
usage of multiple (different) metadataquery sources ; the actual assertion is 
based on the fact that the new metadataquery gives a better estimation for the 
same rel.

the planner switches over to the default metadataquery provider after some 
time; however this switch may cause trouble - as after this switch the already 
cached internal visited nodesets become incorrect.

https://github.com/apache/hive/blob/f49d257c560c81c38259e95023b20c544acb4d10/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L2234




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23552) TestMiniLlapCliDriver.testCliDriver[merge_test_dummy_operator] is unstable

2020-05-26 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23552:
---

 Summary: 
TestMiniLlapCliDriver.testCliDriver[merge_test_dummy_operator] is unstable
 Key: HIVE-23552
 URL: https://issues.apache.org/jira/browse/HIVE-23552
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23549) TestJdbcWithMiniLlapVectorArrow is unstable

2020-05-26 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23549:
---

 Summary: TestJdbcWithMiniLlapVectorArrow is unstable
 Key: HIVE-23549
 URL: https://issues.apache.org/jira/browse/HIVE-23549
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23548) TestActivePassiveHA is unstable

2020-05-25 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23548:
---

 Summary: TestActivePassiveHA is unstable
 Key: HIVE-23548
 URL: https://issues.apache.org/jira/browse/HIVE-23548
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23528) TestJdbcWithServiceDiscovery is unstable

2020-05-21 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23528:
---

 Summary: TestJdbcWithServiceDiscovery is unstable
 Key: HIVE-23528
 URL: https://issues.apache.org/jira/browse/HIVE-23528
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


{code}
Error Message
tExecute
Stacktrace
java.lang.AssertionError: tExecute
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertNotNull(Assert.java:621)
at 
org.apache.hive.jdbc.TestJdbcWithServiceDiscovery.testKillQueryWithDifferentServer(TestJdbcWithServiceDiscovery.java:271)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
at 
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
Standard Outpu
{coide}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23527) CUME_DIST should be threated as a function returning double

2020-05-21 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23527:
---

 Summary: CUME_DIST should be threated as a function returning 
double
 Key: HIVE-23527
 URL: https://issues.apache.org/jira/browse/HIVE-23527
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


right now it is handled as integer in calcite - which might cause some trouble 
(at least for me)

Interestingly enough it seems like only percent_rank has double type

https://github.com/apache/hive/blob/b047cfae872244e769f5d1c3c11811d1c49c19ad/ql/src/java/org/apache/hadoop/hive/ql/parse/type/HiveFunctionHelper.java#L392

since this change goes back to HIVE-9133 and seems to me that removing this 
tweak is just works - I'll submit a patch to remove this conditional



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23525) TestAcidTxnCleanerService is unstable

2020-05-21 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23525:
---

 Summary: TestAcidTxnCleanerService is unstable
 Key: HIVE-23525
 URL: https://issues.apache.org/jira/browse/HIVE-23525
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


from time to time this exception happens

http://34.66.156.144:8080/job/hive-c/7/console

{code}
15:03:41  [INFO] 
15:03:41  [INFO] ---
15:03:41  [INFO]  T E S T S
15:03:41  [INFO] ---
15:03:42  [INFO] Running 
org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
15:04:10  [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time 
elapsed: 25.582 s <<< FAILURE! - in 
org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
15:04:10  [ERROR] 
cleansAllCommittedTxns(org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService)
  Time elapsed: 9.952 s  <<< FAILURE!
15:04:10  java.lang.AssertionError: expected:<6> but was:<7>
15:04:10at 
org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService.cleansAllCommittedTxns(TestAcidTxnCleanerService.java:107)
15:04:10  
15:04:10  [INFO] 
15:04:10  [INFO] Results:
15:04:10  [INFO] 
15:04:10  [ERROR] Failures: 
15:04:10  [ERROR]   TestAcidTxnCleanerService.cleansAllCommittedTxns:107 
expected:<6> but was:<7>
15:04:10  [INFO] 
15:04:10  [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
15:04:10  [INFO] 
15:04:10  [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on 
project hive-standalone-metastore-server: There are test failures.
15:04:10  [ERROR] 
15:04:10  [ERROR] Please refer to 
/home/jenkins/agent/workspace/hive-c/standalone-metastore/metastore-server/target/surefire-reports
 for the individual test results.
15:04:10  [ERROR] Please refer to dump files (if any exist) 
[date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
15:04:10  [ERROR] -> [Help 1]
15:04:10  [ERROR] 
15:04:10  [ERROR] To see the full stack trace of the errors, re-run Maven with 
the -e switch.
15:04:10  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
15:04:10  [ERROR] 
15:04:10  [ERROR] For more information about the errors and possible solutions, 
please read the following articles:
15:04:10  [ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23524) TestNewGetSplitsFormat is unstable

2020-05-21 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23524:
---

 Summary: TestNewGetSplitsFormat is unstable
 Key: HIVE-23524
 URL: https://issues.apache.org/jira/browse/HIVE-23524
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich



http://34.66.156.144:8080/job/hive-c/5/console

{code}
12:05:16  [INFO] ---
12:05:16  [INFO]  T E S T S
12:05:16  [INFO] ---
12:05:16  [INFO] Running org.apache.hive.jdbc.TestNewGetSplitsFormat
12:12:38  [ERROR] Tests run: 9, Failures: 0, Errors: 2, Skipped: 0, Time 
elapsed: 433.083 s <<< FAILURE! - in org.apache.hive.jdbc.TestNewGetSplitsFormat
12:12:38  [ERROR] 
testLlapInputFormatEndToEnd(org.apache.hive.jdbc.TestNewGetSplitsFormat)  Time 
elapsed: 82.662 s  <<< ERROR!
12:12:38  java.io.IOException: java.sql.SQLException: Error while cleaning up 
the server resources
12:12:38at 
org.apache.hadoop.hive.llap.LlapBaseInputFormat.getSplits(LlapBaseInputFormat.java:350)
12:12:38at 
org.apache.hadoop.hive.llap.LlapArrowRowInputFormat.getSplits(LlapArrowRowInputFormat.java:54)
12:12:38at 
org.apache.hive.jdbc.TestNewGetSplitsFormat.processQuery(TestNewGetSplitsFormat.java:87)
12:12:38at 
org.apache.hive.jdbc.BaseJdbcWithMiniLlap.processQuery(BaseJdbcWithMiniLlap.java:676)
12:12:38at 
org.apache.hive.jdbc.BaseJdbcWithMiniLlap.testLlapInputFormatEndToEnd(BaseJdbcWithMiniLlap.java:223)
12:12:38at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
12:12:38at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
12:12:38at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
12:12:38at java.lang.reflect.Method.invoke(Method.java:498)
12:12:38at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
12:12:38at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
12:12:38at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
12:12:38at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
12:12:38at 
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
12:12:38  Caused by: java.sql.SQLException: Error while cleaning up the server 
resources
12:12:38at 
org.apache.hive.jdbc.HiveConnection.close(HiveConnection.java:1018)
12:12:38at 
org.apache.hadoop.hive.llap.LlapBaseInputFormat.getSplits(LlapBaseInputFormat.java:342)
12:12:38... 13 more
12:12:38  Caused by: org.apache.thrift.transport.TTransportException: 
java.net.SocketTimeoutException: Read timed out
12:12:38at 
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
12:12:38at 
org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
12:12:38at 
org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:374)
12:12:38at 
org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:451)
12:12:38at 
org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:433)
12:12:38at 
org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
12:12:38at 
org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
12:12:38at 
org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
12:12:38at 
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
12:12:38at 
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
12:12:38at 
org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_CloseSession(TCLIService.java:199)
12:12:38at 
org.apache.hive.service.rpc.thrift.TCLIService$Client.CloseSession(TCLIService.java:186)
12:12:38at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
12:12:38at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
12:12:38at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
12:12:38at java.lang.reflect.Method.invoke(Method.java:498)
12:12:38at 
org.apache.hive.jdbc.HiveConnection$SynchronizedHandler.invoke(HiveConnection.java:1732)
12:12:38at com.sun.proxy.$Proxy139.CloseSession(Unknown Source)
12:12:38at 
org.apache.hive.jdbc.HiveConnection.close(HiveConnection.java:1016)
12:12:38... 14 more
12:12:38  Caused by: java.net.SocketTimeoutException: Read timed out
12:12:38at java.net.SocketInputStream.socketRead0(Native Method)
12:12:38at 
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
12:12:38at java.net.SocketInput

[jira] [Created] (HIVE-23523) TestTriggersTezSessionPoolManager sometimes exits the JVM

2020-05-21 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23523:
---

 Summary: TestTriggersTezSessionPoolManager sometimes exits the JVM
 Key: HIVE-23523
 URL: https://issues.apache.org/jira/browse/HIVE-23523
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


{code}
12:34:01  [INFO] 
12:34:01  [INFO] ---
12:34:01  [INFO]  T E S T S
12:34:01  [INFO] ---
12:34:01  [INFO] Running org.apache.hive.jdbc.TestTriggersTezSessionPoolManager
12:44:09  [INFO] 
12:44:09  [INFO] Results:
12:44:09  [INFO] 
12:44:09  [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
12:44:09  [INFO] 
12:44:09  [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on 
project hive-it-unit: There are test failures.
12:44:09  [ERROR] 
12:44:09  [ERROR] Please refer to 
/home/jenkins/agent/workspace/hive-c/itests/hive-unit/target/surefire-reports 
for the individual test results.
12:44:09  [ERROR] Please refer to dump files (if any exist) 
[date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
12:44:09  [ERROR] ExecutionException The forked VM terminated without properly 
saying goodbye. VM crash or System.exit called?
12:44:09  [ERROR] Command was /bin/sh -c cd 
/home/jenkins/agent/workspace/hive-c/itests/hive-unit && 
/usr/lib/jvm/zulu-8-amd64/jre/bin/java -Xmx2048m -jar 
/home/jenkins/agent/workspace/hive-c/itests/hive-unit/target/surefire/surefirebooter2756019330500992692.jar
 /home/jenkins/agent/workspace/hive-c/itests/hive-unit/target/surefire 
2020-05-18T10-33-57_988-jvmRun1 surefire6986381633375506092tmp 
surefire_08030518552390892289tmp
12:44:09  [ERROR] Process Exit Code: 0
12:44:09  [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: 
ExecutionException The forked VM terminated without properly saying goodbye. VM 
crash or System.exit called?
12:44:09  [ERROR] Command was /bin/sh -c cd 
/home/jenkins/agent/workspace/hive-c/itests/hive-unit && 
/usr/lib/jvm/zulu-8-amd64/jre/bin/java -Xmx2048m -jar 
/home/jenkins/agent/workspace/hive-c/itests/hive-unit/target/surefire/surefirebooter2756019330500992692.jar
 /home/jenkins/agent/workspace/hive-c/itests/hive-unit/target/surefire 
2020-05-18T10-33-57_988-jvmRun1 surefire6986381633375506092tmp 
surefire_08030518552390892289tmp
12:44:09  [ERROR] Process Exit Code: 0
12:44:09  [ERROR]   at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:494)
12:44:09  [ERROR]   at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:441)
12:44:09  [ERROR]   at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:293)
12:44:09  [ERROR]   at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
12:44:09  [ERROR]   at 
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1149)
12:44:09  [ERROR]   at 
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:978)
12:44:09  [ERROR]   at 
org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:854)
12:44:09  [ERROR]   at 
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
12:44:09  [ERROR]   at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
12:44:09  [ERROR]   at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
12:44:09  [ERROR]   at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
12:44:09  [ERROR]   at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
12:44:09  [ERROR]   at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
12:44:09  [ERROR]   at 
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
12:44:09  [ERROR]   at 
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
12:44:09  [ERROR]   at 
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
12:44:09  [ERROR]   at 
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
12:44:09  [ERROR]   at 
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
12:44:09  [ERROR]   at 
org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
12:44:09  [ERROR]   at 
org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
12:44:09  [ERROR]   at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
12:44:09  [ERROR]   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
12:44:09  [ERR

[jira] [Created] (HIVE-23482) Use junit5 to execute tests

2020-05-16 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23482:
---

 Summary: Use junit5 to execute tests
 Key: HIVE-23482
 URL: https://issues.apache.org/jira/browse/HIVE-23482
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23481) Provide a way to include/exclude a set of tests

2020-05-16 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23481:
---

 Summary: Provide a way to include/exclude a set of tests
 Key: HIVE-23481
 URL: https://issues.apache.org/jira/browse/HIVE-23481
 Project: Hive
  Issue Type: Sub-task
  Components: Testing Infrastructure
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


goal here would be to have some categories to enable/disable a set of tests
* metastore
* replication




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23462) Add option to rewrite NTILE to sketch functions

2020-05-13 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23462:
---

 Summary: Add option to rewrite NTILE to sketch functions
 Key: HIVE-23462
 URL: https://issues.apache.org/jira/browse/HIVE-23462
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23460) Add qoption to disable qtests

2020-05-13 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23460:
---

 Summary: Add qoption to disable qtests
 Key: HIVE-23460
 URL: https://issues.apache.org/jira/browse/HIVE-23460
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich
 Attachments: HIVE-23460.01.patch

instead other ways to exclude them... (testconfiguration.properties; 
CliConfig#excludeQuery)

{code}
--! qt:disabled:reason
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23450) TestMiniDruidCliDriver#testCliDriver[druid_materialized_view_rewrite_ssb]

2020-05-12 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23450:
---

 Summary: 
TestMiniDruidCliDriver#testCliDriver[druid_materialized_view_rewrite_ssb] 
 Key: HIVE-23450
 URL: https://issues.apache.org/jira/browse/HIVE-23450
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich





https://builds.apache.org/job/PreCommit-HIVE-Build/22267/testReport/junit/org.apache.hadoop.hive.cli/TestMiniDruidCliDriver/testCliDriver_druid_materialized_view_rewrite_ssb_/

error tends to be:
{code}
[ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
952.613 s <<< FAILURE! - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
[ERROR] 
testCliDriver[druid_materialized_view_rewrite_ssb](org.apache.hadoop.hive.cli.TestMiniDruidCliDriver)
  Time elapsed: 31.851 s  <<< FAILURE!
java.lang.AssertionError: 
Client execution failed with error code = 1 
running 

SELECT MONTH(`__time`) AS `mn___time_ok`,
CAST((MONTH(`__time`) - 1) / 3 + 1 AS BIGINT) AS `qr___time_ok`,
SUM(1) AS `sum_number_of_records_ok`,
YEAR(`__time`) AS `yr___time_ok`
FROM `ssb_mv_druid_100`
GROUP BY MONTH(`__time`),
CAST((MONTH(`__time`) - 1) / 3 + 1 AS BIGINT),
YEAR(`__time`) 
fname=druid_materialized_view_rewrite_ssb.q
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions

2020-05-11 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23434:
---

 Summary: Add option to rewrite PERCENTILE_CONT to sketch functions
 Key: HIVE-23434
 URL: https://issues.apache.org/jira/browse/HIVE-23434
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23404) Schedules in the past should be accepted

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23404:
---

 Summary: Schedules in the past should be accepted
 Key: HIVE-23404
 URL: https://issues.apache.org/jira/browse/HIVE-23404
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


Right now if a schedule in the past is set ; an Exception is thrown.

However this behaviour can be argued...consider that at some point a schedule 
is created to run in say every day in 2020's december(but not after it). When 
the creation happens it's "okay"...but in case we reach 2021 it will be 
considered invalid..because there is no future execution



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23402) strict_pruning_2 is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23402:
---

 Summary: strict_pruning_2 is unstable
 Key: HIVE-23402
 URL: https://issues.apache.org/jira/browse/HIVE-23402
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


this qtest gets a slightly different exception in case it comes thru 
materialized view stuff...
somehow the MV registry retains some old view definition from a different 
test



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23401) TestDbNotificationListener#sqlInsertPartition is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23401:
---

 Summary: TestDbNotificationListener#sqlInsertPartition is unstable
 Key: HIVE-23401
 URL: https://issues.apache.org/jira/browse/HIVE-23401
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich



it seems 1 event is missing at this point:
https://github.com/apache/hive/blob/f349b8c45d94f67add3ec1c64eea6ac88f4e5ed4/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java#L1528

https://github.com/apache/hive/blob/f349b8c45d94f67add3ec1c64eea6ac88f4e5ed4/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java#L323


http://34.66.156.144:8080/job/hive-precommit/view/change-requests/job/PR-7/54/testReport/org.apache.hive.hcatalog.listener/TestDbNotificationListener/Testing___split_06___Archive___sqlInsertPartition/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23400) TransactionalKafkaWriterTest#writerFencedOut is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23400:
---

 Summary: TransactionalKafkaWriterTest#writerFencedOut is unstable
 Key: HIVE-23400
 URL: https://issues.apache.org/jira/browse/HIVE-23400
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


sometimes it fails because the expected exception is not thrown...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23399) TestSessionManagerMetrics#testAbandonedSessionMetrics is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23399:
---

 Summary: TestSessionManagerMetrics#testAbandonedSessionMetrics is 
unstable
 Key: HIVE-23399
 URL: https://issues.apache.org/jira/browse/HIVE-23399
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich



it seems there were already some tries to fix this:
https://github.com/apache/hive/blob/f349b8c45d94f67add3ec1c64eea6ac88f4e5ed4/service/src/test/org/apache/hive/service/cli/session/TestSessionManagerMetrics.java#L390



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23398) TestBeeLineWithArgs#testRowsAffected is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23398:
---

 Summary: TestBeeLineWithArgs#testRowsAffected is unstable
 Key: HIVE-23398
 URL: https://issues.apache.org/jira/browse/HIVE-23398
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


the common thing when it fails is that the execution time of the last command 
always took >30 seconds

{code}
20/05/06 15:45:17 INFO ql.Driver: Completed executing 
command(queryId=jenkins_20200506154436_3a57c3b5-2aca-452b-b948-bd2d457f7ee4); 
Time taken: 39.253 seconds
20/05/06 15:45:17 DEBUG log.PerfLogger: 
20/05/06 15:45:17 DEBUG log.PerfLogger: 
20/05/06 15:45:17 DEBUG ql.Driver: Shutting down query insert into new_table 
values (1)
Unknown HS2 problem when communicating with Thrift server.
Error: Failed to wait for operation to complete (state=08S01,code=0)
Closing: 0: jdbc:hive2://localhost:33585/
 should contain .*1 row affected.*
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23397) TransactionalKafkaWriterTest#writerFencedOut is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23397:
---

 Summary: TransactionalKafkaWriterTest#writerFencedOut is unstable
 Key: HIVE-23397
 URL: https://issues.apache.org/jira/browse/HIVE-23397
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich


sometimes it fails with that it didn't got the expected exception



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23396) TestHiveRemote is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23396:
---

 Summary: TestHiveRemote is unstable
 Key: HIVE-23396
 URL: https://issues.apache.org/jira/browse/HIVE-23396
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


sometimes it seems to fail ; there were also some derby exceptions earlier

the test itself has a @Before annotated setUp ; which starts a metastore - I 
think this should be done only once ; so @BeforeClass might be a better 
option...

{code}
org.apache.hadoop.hive.metastore.api.NoSuchObjectException: database 
hive.default
at 
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:719) 
~[hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_252]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_252]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_252]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
at 
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) 
~[hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy41.getDatabase(Unknown Source) ~[?:?]
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:2076)
 ~[hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_req(HiveMetaStore.java:2324)
 [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_252]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_252]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_252]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
 [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
 [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy43.create_table_req(Unknown Source) [?:?]
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_req.getResult(ThriftHiveMetastore.java:16497)
 [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_req.getResult(ThriftHiveMetastore.java:16481)
 [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
[libthrift-0.9.3-1.jar:0.9.3-1]
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
 [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
 [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method) 
~[?:1.8.0_252]
at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_252]
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
 [hadoop-common-3.1.0.jar:?]
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
 [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
 [libthrift-0.9.3-1.jar:0.9.3-1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[?:1.8.0_252]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-05-06T17:04:30,243 ERROR [pool-9-thread-3] metastore.RetryingHMSHandler: 
InvalidObjectException(message:database hive.default)
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_req(HiveMetaStore.java:2329)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.ja

[jira] [Created] (HIVE-23395) TestScheduledReplicationScenarios#testAcidTablesReplLoadBootstrapIncr is unstable

2020-05-07 Thread Zoltan Haindrich (Jira)
Zoltan Haindrich created HIVE-23395:
---

 Summary: 
TestScheduledReplicationScenarios#testAcidTablesReplLoadBootstrapIncr is 
unstable
 Key: HIVE-23395
 URL: https://issues.apache.org/jira/browse/HIVE-23395
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


sometimes it fails with below exception

{code}
Error Message
expected:<4> but was:<3>
Stacktrace
java.lang.AssertionError: expected:<4> but was:<3>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at 
org.apache.hadoop.hive.ql.parse.WarehouseInstance.verifyResults(WarehouseInstance.java:395)
at 
org.apache.hadoop.hive.ql.parse.TestScheduledReplicationScenarios.testAcidTablesReplLoadBootstrapIncr(TestScheduledReplicationScenarios.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
at 
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
Standard Output
Formatting using clusterid: testClusterID
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


<    1   2   3   4   5   6   7   >