date:20230319

[jira] [Created] (HIVE-27154) Fix testBootstrapReplLoadRetryAfterFailureForPartitions

2023-03-19 Thread Vihang Karajgaonkar (Jira)

Vihang Karajgaonkar created HIVE-27154:
--

 Summary: Fix testBootstrapReplLoadRetryAfterFailureForPartitions
 Key: HIVE-27154
 URL: https://issues.apache.org/jira/browse/HIVE-27154
 Project: Hive
  Issue Type: Sub-task
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar


`testBootstrapReplLoadRetryAfterFailureForPartitions` has been failing on 
branch-3

 

http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4067/12/tests



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [EXTERNAL] Re: Branch-3 backports and build stability

2023-03-19 Thread Aman Raj

Hi Vihang/community,

Found the ticket which broke mm_all.q. This issue comes because of HIVE-20182. 
Works in my local and on the Jenkins pipeline as well. Link : 
https://github.com/apache/hive/pull/4127 Reverting this commit for now.

Thanks,
Aman.

From: Aman Raj 
Sent: Monday, March 20, 2023 8:28 AM
To: dev@hive.apache.org 
Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability

Sure Vihang, will look at the other ones. You can pick this up.

Thanks,
Aman.

Get Outlook for 
Android

From: vihang karajgaonkar 
Sent: Monday, March 20, 2023 7:58:48 AM
To: dev@hive.apache.org 
Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability

I think we should revert offending commits first to unblock the branch. We
can create followup tickets to determine if these fixes are blockers for
3.2 release and if yes, we should merge them the right way with a green
test run. Fixing forward always comes with the risk that it introduces new
test failures.

Thanks for all your efforts on this Aman.

I can take a look at testBootstrapReplLoadRetryAfterFailureForPartitions if
you haven’t already started on it.

Thanks,
Vihang

On Sun, Mar 19, 2023 at 10:09 PM Aman Raj 
wrote:

> Hi Vihang/community,
>
> Thanks a lot Vihang for working on the major test failure. This blocked
> more than 35 test cases. Now we are down to the final 4 failures. I have
> analyzed some of them and here they are  (Link :
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fblue%2Forganizations%2Fjenkins%2Fhive-precommit%2Fdetail%2FPR-4067%2F12%2Ftests=05%7C01%7Crajaman%40microsoft.com%7Cd6ea3e0148854cd10d8208db28eefb74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148779155151838%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=u9LTxbyc%2BPjK6gWdIwVnPcDK02TIp%2FsnNgeNu%2BvHBZM%3D=0)
> :
>
>   1.
> multi_in_clause - This was committed in HIVE-21685 without validating the
> scenario.
> This fails because Hive is not able to parse
> explain cbo
> select * from very_simple_table_for_in_test where name IN('g','r') AND
> name IN('a','b')
> If we want this to work, I am able to do it in my local. We have 2 options
> :
> a. Either revert HIVE-21685 since this scenario was not validated back
> then before adding this test.
> b. This fix was present in
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-20718=05%7C01%7Crajaman%40microsoft.com%7Cd6ea3e0148854cd10d8208db28eefb74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148779155151838%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=qOOWnfO8K8jXEegm7uZ6tBlhAh2QtmcwTB%2Bu5r5QDSY%3D=0
>  but to cherry pick this
> we need to cherry pick 
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040=05%7C01%7Crajaman%40microsoft.com%7Cd6ea3e0148854cd10d8208db28eefb74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148779155151838%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=2b62n0%2F0OlIskXA37qoHzRRyBcEaR%2BAhrjN9J1yAfxs%3D=0
> since 
> HIVE-20718
>  has a
> lot of merge conflicts with  HIVE-17040<
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040=05%7C01%7Crajaman%40microsoft.com%7Cd6ea3e0148854cd10d8208db28eefb74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148779155151838%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=2b62n0%2F0OlIskXA37qoHzRRyBcEaR%2BAhrjN9J1yAfxs%3D=0>.
>  But after cherry
> picking these we have other failures to fix.
>   2.
> current_date_timestamp.q - This breaking change was committed in
> HIVE-21388 without validation.
> The failure is because again Hive is not able to parse
> explain cbo select current_timestamp() from alltypesorc
> The solution or revert option is same as point 1.
>   3.
> testBootstrapReplLoadRetryAfterFailureForPartitions() - This I have not
> investigated till now.

[jira] [Created] (HIVE-27153) Revert "HIVE-20182: Backport HIVE-20067 to branch-3"

2023-03-19 Thread Aman Raj (Jira)

Aman Raj created HIVE-27153:
---

 Summary: Revert "HIVE-20182: Backport HIVE-20067 to branch-3"
 Key: HIVE-27153
 URL: https://issues.apache.org/jira/browse/HIVE-27153
 Project: Hive
  Issue Type: Sub-task
Reporter: Aman Raj
Assignee: Aman Raj


The mm_all.q test is failing because of this commit. This commit was not 
validated before committing.

There is no stack trace for this exception. Link to the exception : 
[http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4126/2/tests]

 
{code:java}
java.lang.AssertionError: Client execution failed with error code = 1 running 
"insert into table part_mm_n0 partition(key_mm=455) select key from 
intermediate_n0" fname=mm_all.q See ./ql/target/tmp/log/hive.log or 
./itests/qtest/target/tmp/log/hive.log, or check ./ql/target/surefire-reports 
or ./itests/qtest/target/surefire-reports/ for specific test cases logs.at 
org.junit.Assert.fail(Assert.java:88)at 
org.apache.hadoop.hive.ql.QTestUtil.failed(QTestUtil.java:2232)  at 
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:180)
 at 
org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)   at 
org.apache.hadoop.hive.cli.split1.TestMiniLlapCliDriver.testCliDriver(TestMiniLlapCliDriver.java:62)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)   
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [EXTERNAL] Re: Branch-3 backports and build stability

2023-03-19 Thread Aman Raj

Sure Vihang, will look at the other ones. You can pick this up.

Thanks,
Aman.

Get Outlook for Android

From: vihang karajgaonkar 
Sent: Monday, March 20, 2023 7:58:48 AM
To: dev@hive.apache.org 
Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability

I think we should revert offending commits first to unblock the branch. We
can create followup tickets to determine if these fixes are blockers for
3.2 release and if yes, we should merge them the right way with a green
test run. Fixing forward always comes with the risk that it introduces new
test failures.

Thanks for all your efforts on this Aman.

I can take a look at testBootstrapReplLoadRetryAfterFailureForPartitions if
you haven’t already started on it.

Thanks,
Vihang

On Sun, Mar 19, 2023 at 10:09 PM Aman Raj 
wrote:

> Hi Vihang/community,
>
> Thanks a lot Vihang for working on the major test failure. This blocked
> more than 35 test cases. Now we are down to the final 4 failures. I have
> analyzed some of them and here they are  (Link :
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fblue%2Forganizations%2Fjenkins%2Fhive-precommit%2Fdetail%2FPR-4067%2F12%2Ftests=05%7C01%7Crajaman%40microsoft.com%7C3c77d352209146ba91ec08db28eae05e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148761521049046%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=ju9ucPcRMofa7DERJyURmawbC5J3oIiiGOKpqFdXPG8%3D=0)
> :
>
>   1.
> multi_in_clause - This was committed in HIVE-21685 without validating the
> scenario.
> This fails because Hive is not able to parse
> explain cbo
> select * from very_simple_table_for_in_test where name IN('g','r') AND
> name IN('a','b')
> If we want this to work, I am able to do it in my local. We have 2 options
> :
> a. Either revert HIVE-21685 since this scenario was not validated back
> then before adding this test.
> b. This fix was present in
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-20718=05%7C01%7Crajaman%40microsoft.com%7C3c77d352209146ba91ec08db28eae05e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148761521049046%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=B%2FEBhlFoOCCxepgG4dfYuTZhExHcIHBU19%2BvVYiOFhY%3D=0
>  but to cherry pick this
> we need to cherry pick 
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040=05%7C01%7Crajaman%40microsoft.com%7C3c77d352209146ba91ec08db28eae05e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148761521049046%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=NrdNa%2FV%2BHoFU757IV380iIoAnAQpBdAmnOhc9Iy41gE%3D=0
> since 
> HIVE-20718
>  has a
> lot of merge conflicts with  HIVE-17040<
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040=05%7C01%7Crajaman%40microsoft.com%7C3c77d352209146ba91ec08db28eae05e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638148761521049046%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=NrdNa%2FV%2BHoFU757IV380iIoAnAQpBdAmnOhc9Iy41gE%3D=0>.
>  But after cherry
> picking these we have other failures to fix.
>   2.
> current_date_timestamp.q - This breaking change was committed in
> HIVE-21388 without validation.
> The failure is because again Hive is not able to parse
> explain cbo select current_timestamp() from alltypesorc
> The solution or revert option is same as point 1.
>   3.
> testBootstrapReplLoadRetryAfterFailureForPartitions() - This I have not
> investigated till now.
>   4.
> mm_all.q - This I have not investigated till now.
>
> Thanks,
> Aman.
> 
> From: vihang karajgaonkar 
> Sent: Friday, March 17, 2023 8:42 PM
> To: dev@hive.apache.org 
> Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability
>
> Just wanted to close the loop on the TestMiniSparkOnYarnCliDriver test
> failures. We will be able to re-enable most of them back on branch-3. The
> ones which were disabled are being tracked separately in a different ticket
> <
>

Re: [EXTERNAL] Re: Branch-3 backports and build stability

2023-03-19 Thread vihang karajgaonkar

I think we should revert offending commits first to unblock the branch. We
can create followup tickets to determine if these fixes are blockers for
3.2 release and if yes, we should merge them the right way with a green
test run. Fixing forward always comes with the risk that it introduces new
test failures.

Thanks for all your efforts on this Aman.

I can take a look at testBootstrapReplLoadRetryAfterFailureForPartitions if
you haven’t already started on it.

Thanks,
Vihang

On Sun, Mar 19, 2023 at 10:09 PM Aman Raj 
wrote:

> Hi Vihang/community,
>
> Thanks a lot Vihang for working on the major test failure. This blocked
> more than 35 test cases. Now we are down to the final 4 failures. I have
> analyzed some of them and here they are  (Link :
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4067/12/tests)
> :
>
>   1.
> multi_in_clause - This was committed in HIVE-21685 without validating the
> scenario.
> This fails because Hive is not able to parse
> explain cbo
> select * from very_simple_table_for_in_test where name IN('g','r') AND
> name IN('a','b')
> If we want this to work, I am able to do it in my local. We have 2 options
> :
> a. Either revert HIVE-21685 since this scenario was not validated back
> then before adding this test.
> b. This fix was present in
> https://issues.apache.org/jira/browse/HIVE-20718 but to cherry pick this
> we need to cherry pick https://issues.apache.org/jira/browse/HIVE-17040
> since HIVE-20718 has a
> lot of merge conflicts with  HIVE-17040<
> https://issues.apache.org/jira/browse/HIVE-17040>. But after cherry
> picking these we have other failures to fix.
>   2.
> current_date_timestamp.q - This breaking change was committed in
> HIVE-21388 without validation.
> The failure is because again Hive is not able to parse
> explain cbo select current_timestamp() from alltypesorc
> The solution or revert option is same as point 1.
>   3.
> testBootstrapReplLoadRetryAfterFailureForPartitions() - This I have not
> investigated till now.
>   4.
> mm_all.q - This I have not investigated till now.
>
> Thanks,
> Aman.
> 
> From: vihang karajgaonkar 
> Sent: Friday, March 17, 2023 8:42 PM
> To: dev@hive.apache.org 
> Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability
>
> Just wanted to close the loop on the TestMiniSparkOnYarnCliDriver test
> failures. We will be able to re-enable most of them back on branch-3. The
> ones which were disabled are being tracked separately in a different ticket
> <
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-27146=05%7C01%7Crajaman%40microsoft.com%7Cfe96faae91f8418ecaa108db26fa0a5e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638146627636747901%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=C19is4AtBNH04Dm1F1bwp4wVw6erFn736e47p6STrzE%3D=0>
> but they don't look like
> a blocker.
>
> Hi Aman,
>
> Do you know how close are we to reopening branch-3?
>
> Thanks,
> Vihang
>
> On Sat, Mar 4, 2023 at 7:23 PM Aman Raj 
> wrote:
>
> > Or you can cd into itests and run the command you are using. Just another
> > way I run.
> >
> > Thanks,
> > Aman.
> > Get Outlook for Android<
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg=05%7C01%7Crajaman%40microsoft.com%7Cfe96faae91f8418ecaa108db26fa0a5e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638146627636747901%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=nAL14KzxAWwQAV5WJmfkBgaJh0M0wPwq5qORrXcQ6fk%3D=0
> >
> > 
> > From: Aman Raj 
> > Sent: Saturday, March 4, 2023 7:20:36 PM
> > To: dev@hive.apache.org 
> > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability
> >
> > Hi Vihang,
> >
> > Thanks a lot for working on this. Can you try using -Pqsplits,itests.
> > Also, I usually give a -o option after doing a clean install.
> >
> > Thanks,
> > Aman.
> >
> > Get Outlook for Android<
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg=05%7C01%7Crajaman%40microsoft.com%7Cfe96faae91f8418ecaa108db26fa0a5e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638146627636747901%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=nAL14KzxAWwQAV5WJmfkBgaJh0M0wPwq5qORrXcQ6fk%3D=0
> >
> >
> > 
> > From: vihang karajgaonkar 
> > Sent: Saturday, 4 March, 2023, 11:35
> > To: dev@hive.apache.org 
> > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability
> >
> > [You don't often get email from vihan...@apache.org. Learn why this is
> > important at https://aka.ms/LearnAboutSenderIdentification ]
> >
> > Just to update on the HoS test failures for
> TestMiniSparkOnYarnCliDriver, I
> > think I was finally able to resolve

[jira] [Created] (HIVE-27152) Revert "Constant UDF is not pushed to JDBCStorage Handler"

2023-03-19 Thread Aman Raj (Jira)

Aman Raj created HIVE-27152:
---

 Summary: Revert "Constant UDF is not pushed to JDBCStorage Handler"
 Key: HIVE-27152
 URL: https://issues.apache.org/jira/browse/HIVE-27152
 Project: Hive
  Issue Type: Sub-task
Reporter: Aman Raj
Assignee: Aman Raj


current_date_timestamp.q - This change was committed in HIVE-21388 without 
validation.
The failure is because again Hive is not able to parse 
explain cbo select current_timestamp() from alltypesorc

 

Exception stack trace :
{code:java}
2023-03-16 04:06:17 Completed running task attempt: 
attempt_1678964507586_0001_175_01_00_02023-03-16 04:06:17 Completed Dag: 
dag_1678964507586_0001_175TRACE StatusLogger Log4jLoggerFactory.getContext() 
found anchor class org.apache.hadoop.hive.ql.exec.OperatorTRACE StatusLogger 
Log4jLoggerFactory.getContext() found anchor class 
org.apache.hadoop.hive.ql.stats.fs.FSStatsPublisherTRACE StatusLogger 
Log4jLoggerFactory.getContext() found anchor class 
org.apache.hadoop.hive.ql.stats.fs.FSStatsAggregatorNoViableAltException(24@[]) 
 at 
org.apache.hadoop.hive.ql.parse.HiveParser.explainStatement(HiveParser.java:1512)
at 
org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1407)   at 
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:230)  at 
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79) at 
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72) at 
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:617)at 
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1854)   at 
org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1801) at 
org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1796) at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
   at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214) at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) at 
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)  at 
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) at 
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335) at 
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1474)  
 at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1448)  
 at 
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
 at 
org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)   at 
org.apache.hadoop.hive.cli.split12.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:62)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)   
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
  at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)  
 at org.junit.rules.RunRules.evaluate(RunRules.java:20) Attachments {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27151) Revert "HIVE-21685 Wrong simplification in query with multiple IN clauses"

2023-03-19 Thread Aman Raj (Jira)

Aman Raj created HIVE-27151:
---

 Summary: Revert "HIVE-21685 Wrong simplification in query with 
multiple IN clauses"
 Key: HIVE-27151
 URL: https://issues.apache.org/jira/browse/HIVE-27151
 Project: Hive
  Issue Type: Sub-task
Reporter: Aman Raj
Assignee: Aman Raj


The multi_in_clause.q fails because Hive is not able to parse 
explain cbo
select * from very_simple_table_for_in_test where name IN('g','r') AND name 
IN('a','b')
If we want this to work, I am able to do it in my local. We have 2 options :
a. Either revert HIVE-21685 since this scenario was not validated back then 
before adding this test.
b. This fix was present in https://issues.apache.org/jira/browse/HIVE-20718 but 
to cherry pick this we need to cherry pick 
https://issues.apache.org/jira/browse/HIVE-17040 since HIVE-20718 has a lot of 
merge conflicts with  HIVE-17040. But after cherry picking these we have other 
failures to fix.
 
I am reverting this ticket for now.

Exception stacktrace :

{code:java}
2023-03-16 12:33:11 Completed running task attempt: 
attempt_1678994907903_0001_185_01_00_02023-03-16 12:33:11 Completed Dag: 
dag_1678994907903_0001_185TRACE StatusLogger Log4jLoggerFactory.getContext() 
found anchor class org.apache.hadoop.hive.ql.exec.OperatorTRACE StatusLogger 
Log4jLoggerFactory.getContext() found anchor class 
org.apache.hadoop.hive.ql.stats.fs.FSStatsPublisherTRACE StatusLogger 
Log4jLoggerFactory.getContext() found anchor class 
org.apache.hadoop.hive.ql.stats.fs.FSStatsAggregatorNoViableAltException(24@[]) 
 at 
org.apache.hadoop.hive.ql.parse.HiveParser.explainStatement(HiveParser.java:1512)
at 
org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1407)   at 
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:230)  at 
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79) at 
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72) at 
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:617)at 
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1854)   at 
org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1801) at 
org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1796) at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
   at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214) at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) at 
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)  at 
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) at 
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335) at 
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1474)  
 at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1448)  
 at 
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
 at 
org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)   at 
org.apache.hadoop.hive.cli.split12.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:62)
 {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [EXTERNAL] Re: Branch-3 backports and build stability

2023-03-19 Thread Aman Raj

Hi Vihang/community,

Thanks a lot Vihang for working on the major test failure. This blocked more 
than 35 test cases. Now we are down to the final 4 failures. I have analyzed 
some of them and here they are  (Link : 
http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4067/12/tests)
 :

  1.
multi_in_clause - This was committed in HIVE-21685 without validating the 
scenario.
This fails because Hive is not able to parse
explain cbo
select * from very_simple_table_for_in_test where name IN('g','r') AND name 
IN('a','b')
If we want this to work, I am able to do it in my local. We have 2 options :
a. Either revert HIVE-21685 since this scenario was not validated back then 
before adding this test.
b. This fix was present in https://issues.apache.org/jira/browse/HIVE-20718 but 
to cherry pick this we need to cherry pick 
https://issues.apache.org/jira/browse/HIVE-17040 since 
HIVE-20718 has a lot of merge 
conflicts with  HIVE-17040. 
But after cherry picking these we have other failures to fix.
  2.
current_date_timestamp.q - This breaking change was committed in HIVE-21388 
without validation.
The failure is because again Hive is not able to parse
explain cbo select current_timestamp() from alltypesorc
The solution or revert option is same as point 1.
  3.
testBootstrapReplLoadRetryAfterFailureForPartitions() - This I have not 
investigated till now.
  4.
mm_all.q - This I have not investigated till now.

Thanks,
Aman.

From: vihang karajgaonkar 
Sent: Friday, March 17, 2023 8:42 PM
To: dev@hive.apache.org 
Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability

Just wanted to close the loop on the TestMiniSparkOnYarnCliDriver test
failures. We will be able to re-enable most of them back on branch-3. The
ones which were disabled are being tracked separately in a different ticket

 but they don't look like
a blocker.

Hi Aman,

Do you know how close are we to reopening branch-3?

Thanks,
Vihang

On Sat, Mar 4, 2023 at 7:23 PM Aman Raj 
wrote:

> Or you can cd into itests and run the command you are using. Just another
> way I run.
>
> Thanks,
> Aman.
> Get Outlook for 
> Android
> 
> From: Aman Raj 
> Sent: Saturday, March 4, 2023 7:20:36 PM
> To: dev@hive.apache.org 
> Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability
>
> Hi Vihang,
>
> Thanks a lot for working on this. Can you try using -Pqsplits,itests.
> Also, I usually give a -o option after doing a clean install.
>
> Thanks,
> Aman.
>
> Get Outlook for 
> Android
>
> 
> From: vihang karajgaonkar 
> Sent: Saturday, 4 March, 2023, 11:35
> To: dev@hive.apache.org 
> Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability
>
> [You don't often get email from vihan...@apache.org. Learn why this is
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Just to update on the HoS test failures for TestMiniSparkOnYarnCliDriver, I
> think I was finally able to resolve them (at least on local). I had to
> revert HIVE-21044 because it was causing OOM for those tests. Also, in
> order for these tests to work we will have to downgrade netty from
> 4.1.69.Final to 4.1.51.Final. I understand that we had upgraded netty from
> 4.1.17.Final to 4.1.69.Final for CVEs but the highest netty version that we
> can support without breaking HoS is 4.1.51.Final. Note that 4.1.51.Final
> includes many of the CVEs which affected 4.1.17.Final so we are still in a
> better place than branch-3.1. Unfortunately, there is no good way to make
> HoS work with a higher netty version so I think we should downgrade the
> netty version to 4.1.51.Final for now and look at more options to upgrade
>

[jira] [Created] (HIVE-27150) Drop single partition can also support direct sql

2023-03-19 Thread Wechar (Jira)

Wechar created HIVE-27150:
-

 Summary: Drop single partition can also support direct sql
 Key: HIVE-27150
 URL: https://issues.apache.org/jira/browse/HIVE-27150
 Project: Hive
  Issue Type: Improvement
  Components: Hive
Reporter: Wechar
Assignee: Wechar


*Background:*
[HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct sql 
for drop_partitions, we can reuse this huge improvement in drop_partition.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27154) Fix testBootstrapReplLoadRetryAfterFailureForPartitions

Re: [EXTERNAL] Re: Branch-3 backports and build stability

[jira] [Created] (HIVE-27153) Revert "HIVE-20182: Backport HIVE-20067 to branch-3"

Re: [EXTERNAL] Re: Branch-3 backports and build stability

Re: [EXTERNAL] Re: Branch-3 backports and build stability

[jira] [Created] (HIVE-27152) Revert "Constant UDF is not pushed to JDBCStorage Handler"

[jira] [Created] (HIVE-27151) Revert "HIVE-21685 Wrong simplification in query with multiple IN clauses"

Re: [EXTERNAL] Re: Branch-3 backports and build stability

[jira] [Created] (HIVE-27150) Drop single partition can also support direct sql

9 matches

Site Navigation

Mail list logo

Footer information