[jira] [Created] (HIVE-25724) Support External only tables for particular databases.
Sai Hemanth Gantasala created HIVE-25724: Summary: Support External only tables for particular databases. Key: HIVE-25724 URL: https://issues.apache.org/jira/browse/HIVE-25724 Project: Hive Issue Type: Improvement Components: HiveServer2, Standalone Metastore Reporter: Sai Hemanth Gantasala Assignee: Sai Hemanth Gantasala Goal: To be able to support disabling of ACID tables for a particular database. Why?: There is a use case to support BDR and hive3 replication engines in parallel. This can be used to avoid the creation of ACID tables in DBs replicated by the BDR engine. (BDR does not have the ability to replicate ACID tables) Some users might want to use this as well for their legacy DBs while they transition to ACID tables in some newer DBs. How? A "special" property to be set on the database. If this property is set, HMS will prevent the creation of ACID tables in this database. Note: # Hive/HMS to implement the enforcement of this property in preventing table creation. # Hive/HMS will not prevent end-users from setting/tampering with this property. -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: hive-exec vs. hive-exec:core
I'm fine as long as we are committed to fixing the shading problems before the release. Ideally I think we should fix the shading problems first and then remove the hive-exec:core jar though (which is I said it's a bit premature to do it now). On Thu, Nov 18, 2021 at 8:28 AM Stamatis Zampetakis wrote: > Hello, > > I don't see any risk committing this right now in master. It will only > affect the new Hive release when and if it ever goes out. > Till then we have plenty of time to fix shading problems and help other > projects migrate to the "recommended" way to use Hive. > > Moreover, I don't know many projects relying on this kind of "double" (core > vs. fat) publication of dependencies. For Hive, it creates additional > maintenance cost and for its users confusion on what they should use. > If for whatever reason, another project does not want to include everything > coming in the fat jar, maven provides ways to do it. I wouldn't recommend > going down this path but there are alternatives. > > Best, > Stamatis > > On Wed, Nov 17, 2021 at 8:15 PM Zoltan Haindrich wrote: > > > > > > > On 11/17/21 7:46 PM, Chao Sun wrote: > > >> We have a working hive-exec jar > > > > > > I'm not sure about this. The issue comes when the fat hive-exec jar > > shades > > > some jars but doesn't relocate them. In this case there is no way for > the > > > downstream projects to resolve the conflict. > > > > Exactly - I think those should be hammered out for good; fix the > > shading/relocation! > > > > > > > > On the Spark side IIUC we had issues with Apache Commons as well as ORC > > > (see HIVE-25317 for an effort on this), and there could be more. Spark > is > > > using Hive 2.3 though but the same applies for master/4.0 if dependency > > > versions differ between Hive and the downstream projects. > > > > This change is only about master - it won't change Hive 2.3. HIVE-25317 > > was for branch-2 as well. > > I've seen a few places wierd stuff because they were not able to use the > > hive-exec jar as-is. > > Folks in the Impala project for example went in a direction to > > re-shade/re-filter the hive-exec jar and relocate some stuff in it - most > > likely because it conflicted with > > their stuff. > > > > > https://github.com/apache/impala/blob/master/java/shaded-deps/hive-exec/pom.xml > > Taking a quick look at https://github.com/apache/spark/pull/33989/files > > it seems like you've also done something similarbut instead of using > > the base artifact; you have > > created a new shader. > > I don't think this better than having an artifact which is simply works > > out-of-the-box. > > > > > > cheers, > > Zoltan > > > > > > > > On Wed, Nov 17, 2021 at 10:35 AM Zoltan Haindrich wrote: > > > > > >> On 11/17/21 7:07 PM, Daniel Fritsi wrote: > > >>> For Oozie we've decided to use fat Jar downstream (Cloudera) as there > > we > > >> have processes to ensure 3rd-party library versions are kept in sync. > > >>> > > >>> Since we don't have such a process in Apache, there we'll continue to > > >> use the core Jar. > > >> > > >> It might be possible to evade some problems by using a 3rd party lib > > >> syncer - but if we've done a good job shading this stuff; it should > not > > >> cause any trouble even in case > > >> other 3rd party stuff is presentbut in any case to check things > out > > >> you will need a Hive release in some form > > >> > > >> cheers, > > >> Zoltan > > >> > > >>> > > >>> Dan > > >>> > > >>> On 2021. 11. 17. 18:50, Chao Sun wrote: > > > the idea is to fix the issues they bump into - because people who > > load > > the jdbc driver may also see those issues. > > > > I don’t get what you mean here, could you elaborate a bit more? > > > > IMO it's a bit premature to do this without a working hive-exec jar > > for > > downstream projects like Spark/Trino/Presto. At the current state > > there > > >> is > > no way to upgrade these projects to use the fat hive-exec jar. > > > > > > > > On Wed, Nov 17, 2021 at 5:47 AM Zoltan Haindrich > wrote: > > > > > Hey all, > > > > > > I wanted to get back to this - but had other things going on. > > > > > > Chao> it is still being used today by some other popular projects > > > the idea is to fix the issues they bump into - because people who > > load > > >> the > > > jdbc driver may also see those issues. > > > > > > Edward> [...] You all must like enjoy shading jars. > > > I totally agree that they may use a shell action as well. > > > I wonder how do you propose to solve issues related to clients > using > > a > > > different version of the guava library? > > > > > > The changes which will remove the core artifact stuff is ready: > > > https://github.com/apache/hive/pull/2648 > > > > > > cheers, > > > Zoltan > > > > > > On 9/21/21 8:23 PM, Edward Capriolo wrote: > > >> recommendation from the Hive team is to use
Re: hive-exec vs. hive-exec:core
Hello, I don't see any risk committing this right now in master. It will only affect the new Hive release when and if it ever goes out. Till then we have plenty of time to fix shading problems and help other projects migrate to the "recommended" way to use Hive. Moreover, I don't know many projects relying on this kind of "double" (core vs. fat) publication of dependencies. For Hive, it creates additional maintenance cost and for its users confusion on what they should use. If for whatever reason, another project does not want to include everything coming in the fat jar, maven provides ways to do it. I wouldn't recommend going down this path but there are alternatives. Best, Stamatis On Wed, Nov 17, 2021 at 8:15 PM Zoltan Haindrich wrote: > > > On 11/17/21 7:46 PM, Chao Sun wrote: > >> We have a working hive-exec jar > > > > I'm not sure about this. The issue comes when the fat hive-exec jar > shades > > some jars but doesn't relocate them. In this case there is no way for the > > downstream projects to resolve the conflict. > > Exactly - I think those should be hammered out for good; fix the > shading/relocation! > > > > > On the Spark side IIUC we had issues with Apache Commons as well as ORC > > (see HIVE-25317 for an effort on this), and there could be more. Spark is > > using Hive 2.3 though but the same applies for master/4.0 if dependency > > versions differ between Hive and the downstream projects. > > This change is only about master - it won't change Hive 2.3. HIVE-25317 > was for branch-2 as well. > I've seen a few places wierd stuff because they were not able to use the > hive-exec jar as-is. > Folks in the Impala project for example went in a direction to > re-shade/re-filter the hive-exec jar and relocate some stuff in it - most > likely because it conflicted with > their stuff. > > https://github.com/apache/impala/blob/master/java/shaded-deps/hive-exec/pom.xml > Taking a quick look at https://github.com/apache/spark/pull/33989/files > it seems like you've also done something similarbut instead of using > the base artifact; you have > created a new shader. > I don't think this better than having an artifact which is simply works > out-of-the-box. > > > cheers, > Zoltan > > > > > On Wed, Nov 17, 2021 at 10:35 AM Zoltan Haindrich wrote: > > > >> On 11/17/21 7:07 PM, Daniel Fritsi wrote: > >>> For Oozie we've decided to use fat Jar downstream (Cloudera) as there > we > >> have processes to ensure 3rd-party library versions are kept in sync. > >>> > >>> Since we don't have such a process in Apache, there we'll continue to > >> use the core Jar. > >> > >> It might be possible to evade some problems by using a 3rd party lib > >> syncer - but if we've done a good job shading this stuff; it should not > >> cause any trouble even in case > >> other 3rd party stuff is presentbut in any case to check things out > >> you will need a Hive release in some form > >> > >> cheers, > >> Zoltan > >> > >>> > >>> Dan > >>> > >>> On 2021. 11. 17. 18:50, Chao Sun wrote: > > the idea is to fix the issues they bump into - because people who > load > the jdbc driver may also see those issues. > > I don’t get what you mean here, could you elaborate a bit more? > > IMO it's a bit premature to do this without a working hive-exec jar > for > downstream projects like Spark/Trino/Presto. At the current state > there > >> is > no way to upgrade these projects to use the fat hive-exec jar. > > > > On Wed, Nov 17, 2021 at 5:47 AM Zoltan Haindrich wrote: > > > Hey all, > > > > I wanted to get back to this - but had other things going on. > > > > Chao> it is still being used today by some other popular projects > > the idea is to fix the issues they bump into - because people who > load > >> the > > jdbc driver may also see those issues. > > > > Edward> [...] You all must like enjoy shading jars. > > I totally agree that they may use a shell action as well. > > I wonder how do you propose to solve issues related to clients using > a > > different version of the guava library? > > > > The changes which will remove the core artifact stuff is ready: > > https://github.com/apache/hive/pull/2648 > > > > cheers, > > Zoltan > > > > On 9/21/21 8:23 PM, Edward Capriolo wrote: > >> recommendation from the Hive team is to use the hive-exec.jar > >> artifact. > >> > >> You know about 10 years ago. I mentioned that oozie should just use > >> hive-service or hive jdbc. After a big fight where folks kept > >> bringing up > >> concurrency bugs in hive-server-1 my prs were rejected (even though > >> hive > >> server2 would not have these bugs). I still cannot fathom why > someone > > using > >> oozie would want a fat jar of hive (as opposed to hive server or > > hivejdbc) > >> . If I had to do that, i would just use shell action. You all > must
[jira] [Created] (HIVE-25723) Found some typos
Feng created HIVE-25723:
---
Summary: Found some typos
Key: HIVE-25723
URL: https://issues.apache.org/jira/browse/HIVE-25723
Project: Hive
Issue Type: Improvement
Affects Versions: All Versions
Reporter: Feng
Fix For: All Versions
I found some typos in DateUtils.java and
RELEASE_NOTES.txt{color:#172b4d}{{}}{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
