[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=651945=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651945
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 18:51
Start Date: 16/Sep/21 18:51
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r710388364
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
we could relocate/shade away those deps to make it possible for other
projects to use the normal artifact - seems like there is a very good list in
the trino project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651945)
Time Spent: 3h 40m (was: 3.5h)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=651928=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651928
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 18:23
Start Date: 16/Sep/21 18:23
Worklog Time Spent: 10m
Work Description: sunchao commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r710369202
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
@kgyrtkirk Guava is shaded in branch-2.3 via
https://issues.apache.org/jira/browse/HIVE-23980. The issue is, in order for
Spark to use shaded `hive-exec`, Hive will need to relocate more classes and at
the same time making sure it won't break other modules (for instance, if the
shaded class appears in certain API and another module imported the unshaded
version of the class by itself).
Currently we've abandoned this approach and decided to shade the
`hive-exec-core` within Spark itself, following similar approach in Trino (see
https://github.com/trinodb/trino-hive-apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651928)
Time Spent: 3.5h (was: 3h 20m)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=651541=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651541
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 08:42
Start Date: 16/Sep/21 08:42
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r709911738
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
note: on branch2 guava is most likely not properly shaded away HIVE-22126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651541)
Time Spent: 3h 20m (was: 3h 10m)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=651539=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651539
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 08:38
Start Date: 16/Sep/21 08:38
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r709908764
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
we should fix the issues with using the normal `hive-exec` artifact if
there is any - loading the core jar could cause troubles...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651539)
Time Spent: 3h 10m (was: 3h)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=650952=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-650952 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 15/Sep/21 07:31 Start Date: 15/Sep/21 07:31 Worklog Time Spent: 10m Work Description: viirya closed pull request #2459: URL: https://github.com/apache/hive/pull/2459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 650952) Time Spent: 3h (was: 2h 50m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Assignee: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=650951=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-650951 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 15/Sep/21 07:31 Start Date: 15/Sep/21 07:31 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-919772056 Close this as we are taking a different direction to shade dependencies of Hive at Spark. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 650951) Time Spent: 2h 50m (was: 2h 40m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Assignee: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=642455=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642455
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 26/Aug/21 16:59
Start Date: 26/Aug/21 16:59
Worklog Time Spent: 10m
Work Description: sunchao commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r696816642
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
Yea I think so since this PR is trying to shade things from the
`hive-exec-core` I believe?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642455)
Time Spent: 2h 40m (was: 2.5h)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=641468=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641468
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 25/Aug/21 02:53
Start Date: 25/Aug/21 02:53
Worklog Time Spent: 10m
Work Description: viirya commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r695352477
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
@sunchao do we need to have similar change on master first?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 641468)
Time Spent: 2.5h (was: 2h 20m)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=641313=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641313
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 24/Aug/21 20:38
Start Date: 24/Aug/21 20:38
Worklog Time Spent: 10m
Work Description: viirya commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r695185072
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
As more dependencies are relocated here, some modules if they depends on
non-core artifact, will cause class not found error...
The motivation is because we want to use shaded version of hive-exec (i.e.,
w/o classifier) in Spark to make sure it doesn't conflict guava version there.
But there are more dependencies conflict with Spark. We need to relocate these
dependencies too..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 641313)
Time Spent: 2h 20m (was: 2h 10m)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=630512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630512
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 28/Jul/21 13:36
Start Date: 28/Jul/21 13:36
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r678307084
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
branch-2.3 ?
please note that changes should land on master first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 630512)
Time Spent: 2h 10m (was: 2h)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[
https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=630510=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630510
]
ASF GitHub Bot logged work on HIVE-25317:
-
Author: ASF GitHub Bot
Created on: 28/Jul/21 13:32
Start Date: 28/Jul/21 13:32
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2459:
URL: https://github.com/apache/hive/pull/2459#discussion_r678304214
##
File path: llap-server/pom.xml
##
@@ -38,6 +38,7 @@
org.apache.hive
hive-exec
${project.version}
+ core
Review comment:
don't use the core artifact - that's just bad!
what are you trying to achieve here with this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 630510)
Time Spent: 2h (was: 1h 50m)
> Relocate dependencies in shaded hive-exec module
>
>
> Key: HIVE-25317
> URL: https://issues.apache.org/jira/browse/HIVE-25317
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> When we want to use shaded version of hive-exec (i.e., w/o classifier), more
> dependencies conflict with Spark. We need to relocate these dependencies too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=623902=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-623902 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 16/Jul/21 21:28 Start Date: 16/Jul/21 21:28 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-881728274 I see. Yea this is very weird. I suspect there are something else that caused it to fail, for instance, when initializing some of the enum items. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 623902) Time Spent: 1h 50m (was: 1h 40m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=623896=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-623896 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 16/Jul/21 21:11 Start Date: 16/Jul/21 21:11 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-881720606 Yea, I'm trying to deal with `org.apache.hive.org.apache.commons.logging.LogFactory`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 623896) Time Spent: 1h 40m (was: 1.5h) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=623894=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-623894 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 16/Jul/21 21:11 Start Date: 16/Jul/21 21:11 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-881720413 The quoted test failure from "Testing / split-20 / Archive / testCommitCreateTablePlusCommitDropTableWithoutPurge – org.apache.hadoop.hive.druid.TestDruidStorageHandler" in http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-2459/11/tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 623894) Time Spent: 1.5h (was: 1h 20m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=623885=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-623885 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 16/Jul/21 21:03 Start Date: 16/Jul/21 21:03 Worklog Time Spent: 10m Work Description: sunchao edited a comment on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-881716282 I'll take a look @viirya - where did you find this error? do you have a link? I'm seeing these errors in the last run: ``` Caused by: java.lang.ClassNotFoundException: org.apache.hive.org.apache.commons.codec.language.Soundex at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:419) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:352) ... 32 more ``` ``` Caused by: java.lang.ClassNotFoundException: org.apache.hive.org.apache.commons.logging.LogFactory at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:419) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:352) ... 35 more ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 623885) Time Spent: 1h 20m (was: 1h 10m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=623884=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-623884 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 16/Jul/21 21:02 Start Date: 16/Jul/21 21:02 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-881716282 I'll take a look @viirya - where did you find this error? do you have a link? I'm seeing this error in the last run: ``` Caused by: java.lang.ClassNotFoundException: org.apache.hive.org.apache.commons.codec.language.Soundex at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:419) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:352) ... 32 more ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 623884) Time Spent: 1h 10m (was: 1h) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=623877=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-623877 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 16/Jul/21 20:54 Start Date: 16/Jul/21 20:54 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-881712574 I'm not sure why this happens, or if it is caused by this change. `HiveConf` is not from hive-exec (ql), and this doesn't relocate any Hive internal class (we shoudn't do this too). cc @sunchao ``` java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hive.conf.HiveConf$ConfVars at org.apache.hadoop.hive.druid.TestDruidStorageHandler.testCommitCreateTablePlusCommitDropTableWithoutPurge(TestDruidStorageHandler.java:135) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 623877) Time Spent: 1h (was: 50m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622256=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622256 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 21:33 Start Date: 13/Jul/21 21:33 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879417843 @viirya there are still more errors: ``` java.lang.NoClassDefFoundError: org/apache/hive/org/apache/commons/logging/LogFactory at org.apache.hadoop.hive.conf.valcoersion.JavaIOTmpdirVariableCoercion.(JavaIOTmpdirVariableCoercion.java:35) at org.apache.hadoop.hive.conf.SystemVariables.(SystemVariables.java:37) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:3445) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:3409) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:436) at org.apache.hadoop.hive.druid.serde.DruidSerDe.initialize(DruidSerDe.java:113) at org.apache.hadoop.hive.serde2.AbstractSerDe.initialize(AbstractSerDe.java:54) at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533) at org.apache.hadoop.hive.druid.TestDruidSerDe.testDruidDeserializer(TestDruidSerDe.java:510) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) ``` ``` Caused by: java.lang.NoClassDefFoundError: org/apache/hive/org/apache/commons/codec/language/Soundex at org.apache.hadoop.hive.ql.udf.generic.GenericUDFSoundex.(GenericUDFSoundex.java:49) ... 29 more ``` ``` Caused by: java.lang.NoClassDefFoundError: org/apache/hive/org/apache/commons/codec/binary/Base64 at org.apache.hadoop.hive.serde2.lazy.LazyBinary.decodeIfNeeded(LazyBinary.java:58) at org.apache.hadoop.hive.serde2.lazy.LazyBinary.init(LazyBinary.java:50) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.uncheckedGetField(LazyStruct.java:226) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:202) at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:128) at org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldData(UnionStructObjectInspector.java:157) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:95) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:68) at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897) at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:438) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:430) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622256) Time Spent: 50m (was: 40m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622212=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622212 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 20:19 Start Date: 13/Jul/21 20:19 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879374726 Thanks @sunchao. Seems caused by relocating avro, protobuf. Reverted the two relocations and see what's going on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622212) Time Spent: 40m (was: 0.5h) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622117 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 17:28 Start Date: 13/Jul/21 17:28 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879269084 Thanks @viirya for the update! there are still some failures that are related, such as 1. ``` java.lang.NoSuchMethodError: org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol.newReflectiveBlockingService(Lorg/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos$LlapDaemonProtocol$BlockingInterface;)Lcom/google/protobuf/BlockingService; at org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.serviceStart(LlapProtocolServerImpl.java:142) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) at org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:405) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.hive.llap.daemon.MiniLlapCluster.serviceStart(MiniLlapCluster.java:177) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.hive.llap.LlapItUtils.startAndGetMiniLlapCluster(LlapItUtils.java:78) at org.apache.hadoop.hive.ql.QTestUtil.setupMiniCluster(QTestUtil.java:656) at org.apache.hadoop.hive.ql.QTestUtil.(QTestUtil.java:584) at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:65) at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:60) at org.apache.hadoop.hive.util.ElapsedTimeLoggingWrapper.invoke(ElapsedTimeLoggingWrapper.java:33) at org.apache.hadoop.hive.cli.control.CoreCliDriver.beforeClass(CoreCliDriver.java:67) at org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:71) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) ``` 2. ``` java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.avro.AvroSerDe.getSchemaFromCols(Ljava/util/Properties;Ljava/util/List;Ljava/util/List;Ljava/lang/String;)Lorg/apache/avro/Schema; at org.apache.hive.hcatalog.mapreduce.SpecialCases.addSpecialCasesParametersToOutputJobProperties(SpecialCases.java:125) at org.apache.hive.hcatalog.mapreduce.FosterStorageHandler.configureOutputJobProperties(FosterStorageHandler.java:217) at org.apache.hive.hcatalog.common.HCatUtil.configureOutputStorageHandler(HCatUtil.java:498) at org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.setOutput(HCatOutputFormat.java:198) at org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.setOutput(HCatOutputFormat.java:70) at org.apache.hive.hcatalog.pig.HCatStorer.setStoreLocation(HCatStorer.java:191) at org.apache.pig.newplan.logical.visitor.InputOutputFileValidatorVisitor.visit(InputOutputFileValidatorVisitor.java:57) at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52) at org.apache.pig.newplan.logical.relational.LogicalPlan.validate(LogicalPlan.java:212) at org.apache.pig.PigServer$Graph.compile(PigServer.java:1851) at org.apache.pig.PigServer$Graph.access$300(PigServer.java:1527) at org.apache.pig.PigServer.execute(PigServer.java:1440) at
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622097=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622097 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 16:29 Start Date: 13/Jul/21 16:29 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879231076 cc @sunchao Most test failures are passed now. Please check if current CI is okay? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622097) Time Spent: 20m (was: 10m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=620674=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620674 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 08/Jul/21 19:31 Start Date: 08/Jul/21 19:31 Worklog Time Spent: 10m Work Description: viirya opened a new pull request #2459: URL: https://github.com/apache/hive/pull/2459 ### What changes were proposed in this pull request? Trying to relocate dependencies which could conflict Spark. ### Why are the changes needed? When we want to use shaded version of hive-exec (i.e., w/o classifier), more dependencies conflict with Spark. We need to relocate these dependencies too. ### Does this PR introduce _any_ user-facing change? If previously downstream projects rely on included dependencies in shaded release, they might need to explicitly include these dependencies after the relocation here. ### How was this patch tested? CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620674) Remaining Estimate: 0h Time Spent: 10m > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
