[jira] [Updated] (HIVE-28118) Hive Insert Into S3 with Viewfs overload scheme fails with MoveTask Error
[
https://issues.apache.org/jira/browse/HIVE-28118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28118:
--
Labels: pull-request-available (was: )
> Hive Insert Into S3 with Viewfs overload scheme fails with MoveTask Error
> -
>
> Key: HIVE-28118
> URL: https://issues.apache.org/jira/browse/HIVE-28118
> Project: Hive
> Issue Type: Bug
>Reporter: Palakur Eshwitha Sai
>Assignee: Palakur Eshwitha Sai
>Priority: Major
> Labels: pull-request-available
>
> With Viewfs overload scheme enabled in the cluster and mount point configured
> to S3, Hive insert into operation fails with MoveTask error.
> {code:java}
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
> hdfs://s3cluster/warehouse/tablespace/external/hive/sales_by_state/state=CA/.hive-staging_hive_2024-01-05_05-52-09_785_7084847051463431810-8/-ext-10002
> to destination
> hdfs://s3cluster/warehouse/tablespace/external/hive/sales_by_state/state=CA/.hive-staging_hive_2024-01-05_05-52-09_785_7084847051463431810-8/-ext-1
> (state=08S01,code=1)
> Caused by: java.io.FileNotFoundException: No such file or directory:
> s3a://testhadoop/warehouse/tablespace/external/hive/sales_by_state/state=CA/.hive-staging_hive_2024-01-05_05-52-09_785_7084847051463431810-8/-ext-1
>
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2344)
> ~[hadoop-aws-3.2.2.3.2.2.4-8.jar:?]
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2226)
> ~[hadoop-aws-3.2.2.3.2.2.4-8.jar:?]
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2160)
> ~[hadoop-aws-3.2.2.3.2.2.4-8.jar:?]
> at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:888)
> ~[hadoop-common-3.2.2.3.2.2.4-6.jar:?]{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-28129) Prepare statement doesnot report the correct query type information
Ramesh Kumar Thangarajan created HIVE-28129: --- Summary: Prepare statement doesnot report the correct query type information Key: HIVE-28129 URL: https://issues.apache.org/jira/browse/HIVE-28129 Project: Hive Issue Type: Task Reporter: Ramesh Kumar Thangarajan Assignee: Ramesh Kumar Thangarajan Prepare statement doesnot report the correct query type information. It inherits the sql statement type of the subsequent queries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28128) explain reoptimization doesnot report the correct querytype information
Ramesh Kumar Thangarajan created HIVE-28128: --- Summary: explain reoptimization doesnot report the correct querytype information Key: HIVE-28128 URL: https://issues.apache.org/jira/browse/HIVE-28128 Project: Hive Issue Type: Task Reporter: Ramesh Kumar Thangarajan Assignee: Ramesh Kumar Thangarajan explain reoptimization doesnot report the correct querytype information and sometimes result in QUERY as the query type instead of explain -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-24167) TPC-DS query 14 fails while generating plan for the filter
[
https://issues.apache.org/jira/browse/HIVE-24167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828418#comment-17828418
]
Stamatis Zampetakis commented on HIVE-24167:
[~kgyrtkirk] [~okumin] I am not sure if there have been offline discussions
about the best path forward but it would be nice to converge on the topic
because the way it is right now the CTE materialization feature appears quite
broken. If you lean towards #5037, #5077, or another approach let me know and I
will try to help review, brainstorm, etc.
> TPC-DS query 14 fails while generating plan for the filter
> --
>
> Key: HIVE-24167
> URL: https://issues.apache.org/jira/browse/HIVE-24167
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Stamatis Zampetakis
>Assignee: okumin
>Priority: Major
> Labels: hive-4.1.0-must, pull-request-available
>
> TPC-DS query 14 (cbo_query14.q and query4.q) fail with NPE on the metastore
> with the partitioned TPC-DS 30TB dataset while generating the plan for the
> filter.
> The problem can be reproduced using the PR in HIVE-23965.
> The current stacktrace shows that the NPE appears while trying to display the
> debug message but even if this line didn't exist it would fail again later on.
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10867)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11765)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11635)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlanForSubQueryPredicate(SemanticAnalyzer.java:3375)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:3473)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10819)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11765)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11625)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11625)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11635)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12417)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:718)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12519)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:443)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:301)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:171)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:301)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:173)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:414)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:363)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:357)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:129)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:231)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:203)
> at
[jira] [Commented] (HIVE-24167) TPC-DS query 14 fails while generating plan for the filter
[
https://issues.apache.org/jira/browse/HIVE-24167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828414#comment-17828414
]
Stamatis Zampetakis commented on HIVE-24167:
It seems that the same problem affects many more TPC-DS queries when CTE
materialization is enabled.
data/conf/perf/tpcds30tb/tez/hive-site.xml:
{noformat}
hive.optimize.cte.materialize.threshold
1
hive.optimize.cte.materialize.full.aggregate.only
false
{noformat}
The settings above trigger CTE materialization for many more queries. Queries
cbo_query23.q, cbo_query33.q, cbo_query51.q, cbo_query56.q, cbo_query58.q,
cbo_query60.q, cbo_query77.q, and cbo_query97.q all fail with the following
error.
{noformat}
java.lang.RuntimeException: equivalence mapping violation
{noformat}
> TPC-DS query 14 fails while generating plan for the filter
> --
>
> Key: HIVE-24167
> URL: https://issues.apache.org/jira/browse/HIVE-24167
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Stamatis Zampetakis
>Assignee: okumin
>Priority: Major
> Labels: hive-4.1.0-must, pull-request-available
>
> TPC-DS query 14 (cbo_query14.q and query4.q) fail with NPE on the metastore
> with the partitioned TPC-DS 30TB dataset while generating the plan for the
> filter.
> The problem can be reproduced using the PR in HIVE-23965.
> The current stacktrace shows that the NPE appears while trying to display the
> debug message but even if this line didn't exist it would fail again later on.
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10867)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11765)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11635)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlanForSubQueryPredicate(SemanticAnalyzer.java:3375)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:3473)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10819)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11765)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11625)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11625)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11635)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12417)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:718)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12519)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:443)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:301)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:171)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:301)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:173)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:414)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:363)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:357)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:129)
>
[jira] [Created] (HIVE-28127) Exception when rebuilding materialized view with calculated columns on iceberg sources
Krisztian Kasa created HIVE-28127:
-
Summary: Exception when rebuilding materialized view with
calculated columns on iceberg sources
Key: HIVE-28127
URL: https://issues.apache.org/jira/browse/HIVE-28127
Project: Hive
Issue Type: Bug
Components: Iceberg integration, Materialized views
Reporter: Krisztian Kasa
Assignee: Krisztian Kasa
{code}
set hive.support.concurrency=true;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
create external table tbl_ice(a int, b string, c int) stored by iceberg stored
as orc tblproperties ('format-version'='1');
insert into tbl_ice values (1, 'one', 50), (4, 'four', 53), (5, 'five', 54);
create materialized view iceberg_mat2 stored by iceberg stored as orc
tblproperties ('format-version'='2') as
select tbl_ice.b, sum(tbl_ice.c), count(tbl_ice.c), avg(tbl_ice.c)
from tbl_ice
group by tbl_ice.b;
insert into tbl_ice values (1, 'one', 50), (2, 'two', 51), (3, 'three', 52),
(4, 'four', 53), (5, 'five', 54);
alter materialized view iceberg_mat2 rebuild;
{code}
{code}
org.apache.hadoop.hive.ql.parse.SemanticException: Line 0:-1 Invalid column
reference '_c3'
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:13598)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:13540)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4931)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4719)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:11554)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:11496)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12432)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12298)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:634)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13162)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:465)
at
org.apache.hadoop.hive.ql.ddl.view.materialized.alter.rebuild.AlterMaterializedViewRebuildAnalyzer.analyzeInternal(AlterMaterializedViewRebuildAnalyzer.java:178)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
at
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
at
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:229)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
at
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:732)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:702)
at
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:115)
at
org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
at
org.apache.hadoop.hive.cli.TestIcebergLlapLocalCliDriver.testCliDriver(TestIcebergLlapLocalCliDriver.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135)
at
[jira] [Updated] (HIVE-28087) Iceberg: Timestamp partition columns with transforms are not correctly sorted during insert
[
https://issues.apache.org/jira/browse/HIVE-28087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28087:
--
Labels: pull-request-available (was: )
> Iceberg: Timestamp partition columns with transforms are not correctly sorted
> during insert
> ---
>
> Key: HIVE-28087
> URL: https://issues.apache.org/jira/browse/HIVE-28087
> Project: Hive
> Issue Type: Task
>Reporter: Simhadri Govindappa
>Assignee: Simhadri Govindappa
>Priority: Major
> Labels: pull-request-available
> Attachments: query-hive-377.csv
>
>
> Insert into partitioned table fails with the following error if the data is
> not clustered.
> *Using cluster by clause it succeeds :*
> {noformat}
> 0: jdbc:hive2://localhost:10001/> insert into table partition_transform_4
> select t, ts from t1 cluster by ts;
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 1 100
> 0 0
> Reducer 2 .. container SUCCEEDED 1 100
> 0 0
> --
> VERTICES: 02/02 [==>>] 100% ELAPSED TIME: 9.47 s
> --
> INFO : Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode
> INFO : Starting task [Stage-0:MOVE] in serial mode
> INFO : Completed executing
> command(queryId=root_20240222123244_0c448b32-4fd9-420d-be31-e39e2972af82);
> Time taken: 10.534 seconds
> 100 rows affected (10.696 seconds){noformat}
>
> *Without cluster By it fails:*
> {noformat}
> 0: jdbc:hive2://localhost:10001/> insert into table partition_transform_4
> select t, ts from t1;
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 1 100
> 0 0
> Reducer 2container RUNNING 1 010
> 2 0
> --
> VERTICES: 01/02 [=>>-] 50% ELAPSED TIME: 9.53 s
> --
> Caused by: java.lang.IllegalStateException: Incoming records violate the
> writer assumption that records are clustered by spec and by partition within
> each spec. Either cluster the incoming records or switch to fanout writers.
> Encountered records that belong to already closed files:
> partition 'ts_month=2027-03' in spec [
> 1000: ts_month: month(2)
> ]
> at org.apache.iceberg.io.ClusteredWriter.write(ClusteredWriter.java:96)
> at
> org.apache.iceberg.io.ClusteredDataWriter.write(ClusteredDataWriter.java:31)
> at
> org.apache.iceberg.mr.hive.writer.HiveIcebergRecordWriter.write(HiveIcebergRecordWriter.java:53)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:1181)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.process(VectorFileSinkOperator.java:111)
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:919)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:158)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:502)
> ... 20 more{noformat}
>
>
> A simple repro, using the attached csv file:
> [^query-hive-377.csv]
> {noformat}
> create database t3;
> use t3;
> create table vector1k(
> t int,
> si int,
> i int,
> b bigint,
> f float,
> d double,
> dc decimal(38,18),
> bo boolean,
> s string,
> s2 string,
> ts timestamp,
> ts2 timestamp,
> dt date)
> row format delimited fields terminated by ',';
> load data local inpath "/query-hive-377.csv" OVERWRITE into table vector1k;
> select * from vector1k; create table vectortab10k(
> t int,
> si int,
> i int,
> b bigint,
> f float,
> d double,
> dc
[jira] [Updated] (HIVE-28111) FSCountersSummary is only printed when LLAP IO is enabled
[
https://issues.apache.org/jira/browse/HIVE-28111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28111:
--
Labels: pull-request-available (was: )
> FSCountersSummary is only printed when LLAP IO is enabled
> -
>
> Key: HIVE-28111
> URL: https://issues.apache.org/jira/browse/HIVE-28111
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
>
> FSCountersSummary is only printed when LLAP IO is enabled, however, it's not
> LLAP IO specific as far as I know:
> https://github.com/apache/hive/blob/03a76ac70370fb94a78b00496ec511e671c652f2/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java#L446-L449
> it gives useful summary like:
> {code}
> INFO : Scheme: S3A
> INFO :
> --
> INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS
> BYTES_WRITTEN WRITE_OPS
> INFO :
> --
> INFO : Map 1 68.53KB 4 0
> 0B 0
> INFO : Map 11 5.54MB68 0
> 0B 0
> INFO : Map 2238.48MB 480 0
> 0B 0
> INFO : Map 5 10.71MB80 0
> 0B 0
> INFO : Map 6 60.68MB 360 0
> 0B 0
> INFO : Map 8 2.69MB24 0
> 0B 0
> INFO : Map 9120.03MB 480 0
> 0B 0
> INFO : Reducer 10 0B 0 0
> 0B 0
> INFO : Reducer 3 0B 0 0
> 0B 0
> INFO : Reducer 4 0B 0 0
> 0B 0
> INFO : Reducer 7 0B 0 0
> 0B 0
> INFO :
> --
> {code}
> need to check if it prints similar data correctly in case of Tez container
> mode as this is specific to the tez runner callable and the
> TaskCounterUpdater is called from tez codepath
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28126) Use added record count in cost model when rebuilding materialized view stored by iceberg
[
https://issues.apache.org/jira/browse/HIVE-28126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-28126:
--
Summary: Use added record count in cost model when rebuilding materialized
view stored by iceberg (was: Use added row count in cost model when rebuilding
materialized view stored by iceberg)
> Use added record count in cost model when rebuilding materialized view
> stored by iceberg
> -
>
> Key: HIVE-28126
> URL: https://issues.apache.org/jira/browse/HIVE-28126
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
>
> There is a cost based decision when rebuilding materialized views having
> aggregate.
> Currently the cost model has the total count of rows of source table however
> incremental rebuild plan scans only the rows inserted since the last MV
> rebuild.
> The goal is to update the row count in the cost model in case of Iceberg
> source tables with the sum of the {{added-records}} stored in snapshots
> summary since the last Mv rebuild.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-28126) Use added row count in cost model when rebuilding materialized view stored by iceberg
Krisztian Kasa created HIVE-28126:
-
Summary: Use added row count in cost model when rebuilding
materialized view stored by iceberg
Key: HIVE-28126
URL: https://issues.apache.org/jira/browse/HIVE-28126
Project: Hive
Issue Type: Improvement
Components: Iceberg integration, Materialized views
Reporter: Krisztian Kasa
Assignee: Krisztian Kasa
There is a cost based decision when rebuilding materialized views having
aggregate.
Currently the cost model has the total count of rows of source table however
incremental rebuild plan scans only the rows inserted since the last MV rebuild.
The goal is to update the row count in the cost model in case of Iceberg source
tables with the sum of the {{added-records}} stored in snapshots summary since
the last Mv rebuild.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28087) Iceberg: Timestamp partition columns with transforms are not correctly sorted during insert
[
https://issues.apache.org/jira/browse/HIVE-28087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sourabh Badhya updated HIVE-28087:
--
Summary: Iceberg: Timestamp partition columns with transforms are not
correctly sorted during insert (was: Hive Iceberg: Insert into partitioned
table fails if the data is not clustered)
> Iceberg: Timestamp partition columns with transforms are not correctly sorted
> during insert
> ---
>
> Key: HIVE-28087
> URL: https://issues.apache.org/jira/browse/HIVE-28087
> Project: Hive
> Issue Type: Task
>Reporter: Simhadri Govindappa
>Assignee: Simhadri Govindappa
>Priority: Major
> Attachments: query-hive-377.csv
>
>
> Insert into partitioned table fails with the following error if the data is
> not clustered.
> *Using cluster by clause it succeeds :*
> {noformat}
> 0: jdbc:hive2://localhost:10001/> insert into table partition_transform_4
> select t, ts from t1 cluster by ts;
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 1 100
> 0 0
> Reducer 2 .. container SUCCEEDED 1 100
> 0 0
> --
> VERTICES: 02/02 [==>>] 100% ELAPSED TIME: 9.47 s
> --
> INFO : Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode
> INFO : Starting task [Stage-0:MOVE] in serial mode
> INFO : Completed executing
> command(queryId=root_20240222123244_0c448b32-4fd9-420d-be31-e39e2972af82);
> Time taken: 10.534 seconds
> 100 rows affected (10.696 seconds){noformat}
>
> *Without cluster By it fails:*
> {noformat}
> 0: jdbc:hive2://localhost:10001/> insert into table partition_transform_4
> select t, ts from t1;
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 1 100
> 0 0
> Reducer 2container RUNNING 1 010
> 2 0
> --
> VERTICES: 01/02 [=>>-] 50% ELAPSED TIME: 9.53 s
> --
> Caused by: java.lang.IllegalStateException: Incoming records violate the
> writer assumption that records are clustered by spec and by partition within
> each spec. Either cluster the incoming records or switch to fanout writers.
> Encountered records that belong to already closed files:
> partition 'ts_month=2027-03' in spec [
> 1000: ts_month: month(2)
> ]
> at org.apache.iceberg.io.ClusteredWriter.write(ClusteredWriter.java:96)
> at
> org.apache.iceberg.io.ClusteredDataWriter.write(ClusteredDataWriter.java:31)
> at
> org.apache.iceberg.mr.hive.writer.HiveIcebergRecordWriter.write(HiveIcebergRecordWriter.java:53)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:1181)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.process(VectorFileSinkOperator.java:111)
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:919)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:158)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:502)
> ... 20 more{noformat}
>
>
> A simple repro, using the attached csv file:
> [^query-hive-377.csv]
> {noformat}
> create database t3;
> use t3;
> create table vector1k(
> t int,
> si int,
> i int,
> b bigint,
> f float,
> d double,
> dc decimal(38,18),
> bo boolean,
> s string,
> s2 string,
> ts timestamp,
> ts2 timestamp,
> dt date)
> row format delimited fields terminated by ',';
> load data local inpath "/query-hive-377.csv" OVERWRITE into table vector1k;
> select * from vector1k; create table vectortab10k(
> t int,
>
[jira] [Commented] (HIVE-27953) Retire https://apache.github.io sites and remove obsolete content/actions
[ https://issues.apache.org/jira/browse/HIVE-27953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828257#comment-17828257 ] Stamatis Zampetakis commented on HIVE-27953: The obsolete content was removed by [https://github.com/apache/hive/commit/33737a123d929af4d1d4c623c9031c68a316c583] However, before marking this as resolved we should wait for INFRA-25604 to be completed and potentially delete unnecessary branches from the repos. > Retire https://apache.github.io sites and remove obsolete content/actions > - > > Key: HIVE-27953 > URL: https://issues.apache.org/jira/browse/HIVE-27953 > Project: Hive > Issue Type: Task > Components: Documentation >Reporter: Stamatis Zampetakis >Assignee: Simhadri Govindappa >Priority: Major > Labels: pull-request-available > > Currently there are three versions of the Hive website (populated from > different places and in various ways) available online. Below, I outline the > entry point URLs along with the latest commit that lead to the deployment > each version. > ||URL||Commit|| > |https://hive.apache.org/|https://github.com/apache/hive-site/commit/0162552c68006fd30411033d5e6a3d6806026851| > |https://apache.github.io/hive/|https://github.com/apache/hive/commit/1455f6201b0f7b061361bc9acc23cb810ff02483| > |https://apache.github.io/hive-site/|https://github.com/apache/hive-site/commit/95b1c8385fa50c2e59579899d2fd297b8a2ecefd| > People searching online for Hive may end-up in any of the above risking to > see pretty outdated information about the project. > For Hive developers (especially newcomers) it is very difficult to figure out > where they should apply their changes if they want to change something in the > website. Even people experienced with the various offering of ASF and GitHub > may have a hard time figuring things out. > I propose to retire/shutdown all GitHub pages deployments > (https://apache.github.io) and drop all content/branches that are not > relevant for the main website under https://hive.apache.org/. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27953) Retire https://apache.github.io sites and remove obsolete content/actions
[ https://issues.apache.org/jira/browse/HIVE-27953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis reassigned HIVE-27953: -- Assignee: Simhadri Govindappa (was: Stamatis Zampetakis) > Retire https://apache.github.io sites and remove obsolete content/actions > - > > Key: HIVE-27953 > URL: https://issues.apache.org/jira/browse/HIVE-27953 > Project: Hive > Issue Type: Task > Components: Documentation >Reporter: Stamatis Zampetakis >Assignee: Simhadri Govindappa >Priority: Major > Labels: pull-request-available > > Currently there are three versions of the Hive website (populated from > different places and in various ways) available online. Below, I outline the > entry point URLs along with the latest commit that lead to the deployment > each version. > ||URL||Commit|| > |https://hive.apache.org/|https://github.com/apache/hive-site/commit/0162552c68006fd30411033d5e6a3d6806026851| > |https://apache.github.io/hive/|https://github.com/apache/hive/commit/1455f6201b0f7b061361bc9acc23cb810ff02483| > |https://apache.github.io/hive-site/|https://github.com/apache/hive-site/commit/95b1c8385fa50c2e59579899d2fd297b8a2ecefd| > People searching online for Hive may end-up in any of the above risking to > see pretty outdated information about the project. > For Hive developers (especially newcomers) it is very difficult to figure out > where they should apply their changes if they want to change something in the > website. Even people experienced with the various offering of ASF and GitHub > may have a hard time figuring things out. > I propose to retire/shutdown all GitHub pages deployments > (https://apache.github.io) and drop all content/branches that are not > relevant for the main website under https://hive.apache.org/. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28115) Upgrade mockito-core from 3.3.3 to 3.4.4 in standalone-metastore
[ https://issues.apache.org/jira/browse/HIVE-28115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis resolved HIVE-28115. Fix Version/s: 4.1.0 Resolution: Fixed Fixed in [https://github.com/apache/hive/commit/cf49f0eaeadd297e444fe6da42849c1e39bb869d] Thanks for the PR [~Aggarwal_Raghav] ! > Upgrade mockito-core from 3.3.3 to 3.4.4 in standalone-metastore > > > Key: HIVE-28115 > URL: https://issues.apache.org/jira/browse/HIVE-28115 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Minor > Labels: pull-request-available > Fix For: 4.1.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28115) Upgrade mockito-core from 3.3.3 to 3.4.4 in standalone-metastore
[ https://issues.apache.org/jira/browse/HIVE-28115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-28115: --- Summary: Upgrade mockito-core from 3.3.3 to 3.4.4 in standalone-metastore (was: Sync mockito-core version in standalone-metastore with parent pom.xml) > Upgrade mockito-core from 3.3.3 to 3.4.4 in standalone-metastore > > > Key: HIVE-28115 > URL: https://issues.apache.org/jira/browse/HIVE-28115 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27598) Enhance alter table compact to work for partitioned tables without specifying any partition
[
https://issues.apache.org/jira/browse/HIVE-27598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828255#comment-17828255
]
Stamatis Zampetakis commented on HIVE-27598:
The design issues mentioned above were addressed as part of HIVE-27848.
> Enhance alter table compact to work for partitioned tables without specifying
> any partition
> ---
>
> Key: HIVE-27598
> URL: https://issues.apache.org/jira/browse/HIVE-27598
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-beta-1
>
>
> *current implementation:* the below command will throw error
>
> {noformat}
> alter table PARTITIONED_T compact 'minor';{noformat}
> for a partitioned table
> {code:java}
> You must specify a partition to compact for partitioned tables{code}
> {*}Problem{*}: One of the problem could be ,let's say a table contains 1000s
> of partitions and for some issue like files permission automatic compaction
> will mark all or few partitions as failed and never pick for compaction.
> After we see performance getting degraded, we find out that there is a
> problem in compaction due to some issue.
> Now to solve it, we have two approaches, adjust failure thresholds and
> restart HMS to pick the changes which is not going to work for someone who
> cannot restart HMS
> And the second approach is to execute alter table mentioning each partition
> {*}Solution{*}:
> In this Jira we are planning to allow alter table command for a partitioned
> table without specifying any partition name.
> Proposed solution is to list all the partitions and find partitions that are
> eligible to get compacted and put an entry for it in compaction queue
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27848) Refactor Initiator hierarchy into CompactorUtil and fix failure in TestCrudCompactorOnTez
[
https://issues.apache.org/jira/browse/HIVE-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis resolved HIVE-27848.
Fix Version/s: 4.1.0
Resolution: Fixed
Fixed in
https://github.com/apache/hive/commit/df45194268ffb10f9f7aae0ab4e3aec35a7b31d8
Thanks for the PR [~tarak271]!
> Refactor Initiator hierarchy into CompactorUtil and fix failure in
> TestCrudCompactorOnTez
> --
>
> Key: HIVE-27848
> URL: https://issues.apache.org/jira/browse/HIVE-27848
> Project: Hive
> Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> The work started initially to fix the
> TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing.
> However, while changing the code to address the failure, the inheritance
> based design for the Initator that was chosen in HIVE-27598 revealed some
> weaknesses briefly outlined below.
> Due to inheritance the InitiatorBase class becomes a Thread something that
> doesn't really make sense and it comes with additional overhead every time we
> instantiate it. Moreover, the only class that currently extends InitiatorBase
> is the Initiator and it's difficult to imagine how we can make other
> extensions from InitiatorBase; the code becomes complex and any subtle change
> in InitiatorBase may have unpredictable effects on Initiator. Having a "Base"
> class that is not really meant to be extended and no instructions on how to
> do so is problematic.
> For the reasons above the focus of the work changed from just re-enabling the
> test to improving and addressing the shortcomings of the inheritance based
> design of Initiator.
> {noformat}
> cd itest/hive-unit
> mvn test
> -Dtest=TestCrudCompactorOnTez#secondCompactionShouldBeRefusedBeforeEnqueueing
> {noformat}
> {noformat}
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 39.548 s <<< FAILURE! - in
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
> [ERROR]
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing
> Time elapsed: 39.444 s <<< FAILURE!
> org.junit.ComparisonFailure: expected:<...ault.compaction_test[] is refused,
> details...> but was:<...ault.compaction_test[CompactionId: -1] is refused,
> details...>
> at org.junit.Assert.assertEquals(Assert.java:117)
> at org.junit.Assert.assertEquals(Assert.java:146)
> at
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing(TestCrudCompactorOnTez.java:638)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> at
[jira] [Updated] (HIVE-27848) Refactor Initiator hierarchy into CompactorUtil and fix failure in TestCrudCompactorOnTez
[
https://issues.apache.org/jira/browse/HIVE-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated HIVE-27848:
---
Description:
The work started initially to fix the
TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing.
However, while changing the code to address the failure, the inheritance based
design for the Initator that was chosen in HIVE-27598 revealed some weaknesses
briefly outlined below.
Due to inheritance the InitiatorBase class becomes a Thread something that
doesn't really make sense and it comes with additional overhead every time we
instantiate it. Moreover, the only class that currently extends InitiatorBase
is the Initiator and it's difficult to imagine how we can make other extensions
from InitiatorBase; the code becomes complex and any subtle change in
InitiatorBase may have unpredictable effects on Initiator. Having a "Base"
class that is not really meant to be extended and no instructions on how to do
so is problematic.
For the reasons above the focus of the work changed from just re-enabling the
test to improving and addressing the shortcomings of the inheritance based
design of Initiator.
{noformat}
cd itest/hive-unit
mvn test
-Dtest=TestCrudCompactorOnTez#secondCompactionShouldBeRefusedBeforeEnqueueing
{noformat}
{noformat}
[INFO] ---
[INFO] Running org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 39.548
s <<< FAILURE! - in
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
[ERROR]
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing
Time elapsed: 39.444 s <<< FAILURE!
org.junit.ComparisonFailure: expected:<...ault.compaction_test[] is refused,
details...> but was:<...ault.compaction_test[CompactionId: -1] is refused,
details...>
at org.junit.Assert.assertEquals(Assert.java:117)
at org.junit.Assert.assertEquals(Assert.java:146)
at
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing(TestCrudCompactorOnTez.java:638)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
at
[jira] [Updated] (HIVE-27848) Refactor Initiator hierarchy into CompactorUtil and fix failure in TestCrudCompactorOnTez
[
https://issues.apache.org/jira/browse/HIVE-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated HIVE-27848:
---
Summary: Refactor Initiator hierarchy into CompactorUtil and fix failure
in TestCrudCompactorOnTez (was:
TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing fails )
> Refactor Initiator hierarchy into CompactorUtil and fix failure in
> TestCrudCompactorOnTez
> --
>
> Key: HIVE-27848
> URL: https://issues.apache.org/jira/browse/HIVE-27848
> Project: Hive
> Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Major
> Labels: pull-request-available
>
> {noformat}
> cd itest/hive-unit
> mvn test
> -Dtest=TestCrudCompactorOnTez#secondCompactionShouldBeRefusedBeforeEnqueueing
> {noformat}
> {noformat}
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 39.548 s <<< FAILURE! - in
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
> [ERROR]
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing
> Time elapsed: 39.444 s <<< FAILURE!
> org.junit.ComparisonFailure: expected:<...ault.compaction_test[] is refused,
> details...> but was:<...ault.compaction_test[CompactionId: -1] is refused,
> details...>
> at org.junit.Assert.assertEquals(Assert.java:117)
> at org.junit.Assert.assertEquals(Assert.java:146)
> at
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing(TestCrudCompactorOnTez.java:638)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
> at
> org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
> {noformat}
> The problem was discovered while trying to re-enable the tests under
> hive-unit module (HIVE-27846).
--
This message was
[jira] [Commented] (HIVE-28123) Add Generated Protobuf code for 3.24.4 Upgrade
[ https://issues.apache.org/jira/browse/HIVE-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828203#comment-17828203 ] Ayush Saxena commented on HIVE-28123: - Committed to master & branch-4.0. Thanx [~Indhumathi27] for the contribution & [~tanishqchugh] for the review!!! > Add Generated Protobuf code for 3.24.4 Upgrade > -- > > Key: HIVE-28123 > URL: https://issues.apache.org/jira/browse/HIVE-28123 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Labels: pull-request-available > > The Protobuf generated files added as part of HIVE-27845 is not correct. > Looks like the contributor has not upgraded protoc in local before generating > the code. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28123) Add Generated Protobuf code for 3.24.4 Upgrade
[ https://issues.apache.org/jira/browse/HIVE-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-28123. - Fix Version/s: 4.0.0 Resolution: Fixed > Add Generated Protobuf code for 3.24.4 Upgrade > -- > > Key: HIVE-28123 > URL: https://issues.apache.org/jira/browse/HIVE-28123 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > The Protobuf generated files added as part of HIVE-27845 is not correct. > Looks like the contributor has not upgraded protoc in local before generating > the code. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28125) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
[ https://issues.apache.org/jira/browse/HIVE-28125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28125: -- Labels: pull-request-available (was: ) > Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 > --- > > Key: HIVE-28125 > URL: https://issues.apache.org/jira/browse/HIVE-28125 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Priority: Major > Labels: pull-request-available > > PR from dependabot: > https://github.com/apache/hive/pull/5130 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28125) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
[ https://issues.apache.org/jira/browse/HIVE-28125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-28125. - Fix Version/s: 4.1.0 Resolution: Fixed > Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 > --- > > Key: HIVE-28125 > URL: https://issues.apache.org/jira/browse/HIVE-28125 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 4.1.0 > > > PR from dependabot: > https://github.com/apache/hive/pull/5130 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-28125) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
[ https://issues.apache.org/jira/browse/HIVE-28125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828200#comment-17828200 ] Ayush Saxena commented on HIVE-28125: - Committed to master. > Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 > --- > > Key: HIVE-28125 > URL: https://issues.apache.org/jira/browse/HIVE-28125 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Priority: Major > Labels: pull-request-available > > PR from dependabot: > https://github.com/apache/hive/pull/5130 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28125) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
Ayush Saxena created HIVE-28125: --- Summary: Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 Key: HIVE-28125 URL: https://issues.apache.org/jira/browse/HIVE-28125 Project: Hive Issue Type: Improvement Reporter: Ayush Saxena PR from dependabot: https://github.com/apache/hive/pull/5130 -- This message was sent by Atlassian Jira (v8.20.10#820010)
