Maintaining Hive 2 and 3 branches,

Hello Hive users, After attending the Hive meetup yesterday (huge thanks to the organizers!), I thought that perhaps many organizations were maintaining their own Hive 2 and 3 branches by backporting important patches to vanilla Hive. Ideally it would be great if all the important patches were

Re: Hive meetup on March 17

Hey All! We have our first online Hive meetup today! We will start at 5pm UTC for other timezones see on this site: https://www.timeanddate.com/worldclock/meetingdetails.html?year=2021=3=17=17=0=0=50=137=136=70=176 If you don't yet have the meeting url - it will be held in a zoom room at:

Re: Hive meetup on March 17

Hey All! Our meetup is also available as a meetup.com event: https://www.meetup.com/Hive-User-Group-Meeting/events/276886707/ In case you want to add it to the calendar or something... :) cheers, Zoltan On 3/11/21 3:00 PM, Zoltan Haindrich wrote: Hey All! I would like to invite you to our

Hive meetup on March 17

Hey All! I would like to invite you to our (first?) online Hive meetup! It will be held on March 17. 17:00 UTC I'll send out a zoom url before the event starts! The planned topics are accessible here:

Re: [EXTERNAL] Re: Any plan for new hive 3 or 4 release?

"My hope has been that Hive 4.x would be built on Java 11. However, I've hit many stumbling blocks over the past year towards this goal." There is not much value in holding back a release. Funny life story, I work at a bank and there are some people still on Java7 which wasend of life in 2015.

BeeJU 5.0.0 (Hive 3) and 4.0.0 (Hive 2) released

Hello Hive users, We are pleased to announce the 4.0.0 and 5.0.0 releases of BeeJU , a unit testing framework for the Hive Metastore and HiveServer2 which supports JUnit4 and Junit5. The main change in version 5.0.0 is support for Hive 3.1.2 and going

Re: Call for Presentations for ApacheCon 2021 now open

shailendra.mis...@oracle.com On Mon, Mar 8, 2021 at 1:14 PM Rich Bowen wrote: > [Note: You are receiving this because you are subscribed to a users@ > list on one or more Apache Software Foundation projects.] > > The ApacheCon Planners and the Apache Software Foundation are pleased to >

Call for Presentations for ApacheCon 2021 now open

[Note: You are receiving this because you are subscribed to a users@ list on one or more Apache Software Foundation projects.] The ApacheCon Planners and the Apache Software Foundation are pleased to announce that ApacheCon@Home will be held online, September 21-23, 2021. Once again, we’ll be

Re: Does Hive support data encryption?

Not directly. It relies on the underlying storage layer. For example: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html On Tue, Mar 2, 2021 at 6:34 AM qq <987626...@qq.com> wrote: > Hello： > > Does Hive support data encryption? > > Thank

Does Hive support data encryption?

Hello?? Does Hive support data encryption? Thank you

Does Hive support data encryption?

Hello?? Does Hive support data encryption? Thank you

Join Our Virtual Meetup! How to Collaborate with Data Lake Management Communities

Thursday, March 11th, join us at Data Lake Management meetup. Ajay Singh from Databricks and Feng Lu from Google Cloud will talk all about data lakes, Delta Lake, Apache Hive metastore & everything in between Don't forget to join us at on March 11th 10 AM PST Sign up here:

CVE-2020-1926: Timing attack in Cookie signature verification

Description: Apache Hive cookie signature verification used a non constant time comparison which is known to be vulnerable to timing attacks. This could allow recovery of another users cookie signature. The issue was addressed in Apache Hive 2.3.8 This issue is being tracked as HIVE-22708

Re: [EXTERNAL] Re: Any plan for new hive 3 or 4 release?

Hello, My hope has been that Hive 4.x would be built on Java 11. However, I've hit many stumbling blocks over the past year towards this goal. I've been able to make some progress, but several things are still stuck. It mostly stems from the fact that hive has many big-ticket dependencies like

Re: Any plan for new hive 3 or 4 release?

The challenge is the venders. They almost always want to tie a release to some offering of there's. Healthy software is released all the time. Just ship it. Call a vote and propose a release. I'll +1 it if the tests pass! On Friday, February 26, 2021, Michel Sumbul wrote: > It will be

Re: Any plan for new hive 3 or 4 release?

It will be amazing if the community could produce a release every quarter/6months. :-) Le ven. 26 févr. 2021 à 14:30, Edward Capriolo a écrit : > Hive was releasable trunk for the longest time. Facebook days. Then the > big data vendors got more involved. Then it became a pissing match about >

Re: Any plan for new hive 3 or 4 release?

Hive was releasable trunk for the longest time. Facebook days. Then the big data vendors got more involved. Then it became a pissing match about features. This vendor likes tez this vendor dont, this vendor likes hive on spark this one dont. Then this vendor wants to tell everyone hive stinks use

Re: Any plan for new hive 3 or 4 release?

I would frankly feel relieved if hive v3 or v4 have none of the developers that developed and maintained v2. Several wasted hours of my life lead me to this conclusion.  That’s all Hernán Sent from Yahoo Mail for iPad On Monday, February 22, 2021, 8:46 AM, Zoltan Haindrich wrote: Hey

Re: Any plan for new hive 3 or 4 release?

Hi Lee, When I started to work on Hive around 4 years ago, MR was already set as deprecated. So you definitely should scan even older archives. For Iceberg integration, it would be good to have more frequent releases for Hive as well. Thanks, Peter Lee Ming-Ta ezt írta (időpont: 2021. febr.

Avro tables with 5k columns any tips?

Hello all, It has been a long time. I have been forced to use avro and create a table with over 5k columns. It's helluva slow. I warned folks that all the best practices say "dont make a table more than 1k or 2k columns" (impala hive cloudera). No one listened to me, so now the table is a mess.

回覆: Any plan for new hive 3 or 4 release?

Dear all, I probably didn't follow that much and would like to ask if anyone can point me to some resources about the reason to remove MR? Or what kine of keyword to search on Google? Thank you very much! Wish everyone a happy Lunar New Year. 寄件者: Mass Dosage

Re: Any plan for new hive 3 or 4 release?

I would love to see a HIve 3.1 release which is capable of being used on Java 11 like Hive 2 is. What is the main difference going to be between Hive 3 and 4? The removal of MR? On Mon, 22 Feb 2021 at 16:46, Zoltan Haindrich wrote: > Hey Michel! > > Yes it was a long time ago we had a release;

Re: Running Hive 3.1.2 embedded in JVM for testing

Have you tried setting "hive.in.test" to "true"? This should get rid of many of the "table X does not exist" errors you were seeing. I know of two other projects that have upgraded from Hive 2 to 3 that run embedded Metastore Services and/or HS2 instances, the pull requests from these might give

Re: Any plan for new hive 3 or 4 release?

Hey Michel! Yes it was a long time ago we had a release; we have quite a few new features in master. I think we are scaring people for some time now that we will be dropping MR support...I think we should do that. I would really like to see a new Hive release in the near future as well -

Any plan for new hive 3 or 4 release?

Hi Guys, If I'm not wrong, the last release of Hive 3.x is 18 months old. I wanted to ask if you had any roadmap / plan to release a new version of Hive 3.x or Hive 4? Thanks, Michel

Re: Running Hive 3.1.2 embedded in JVM for testing

So, last update on my issues. I have reached a point where the embedded hive server starts up without any exceptions thrown. It seems that I needed to disable direct sql. It's not entirely clear to me what that setting actually does, but it seems to clear up the SQL execution warnings in embedded

Re: Running Hive 3.1.2 embedded in JVM for testing

Correction, there are still exceptions related to the metastore not having schemas created, but they are not keeping the service from starting. Things still seem a little sketchy - this is a lot of exceptions for each start up which makes me worried. I'd love to hear if anyone had any other ideas

Re: Running Hive 3.1.2 embedded in JVM for testing

Hi Stamatis, Thanks for the input, I just tried using a memory database within derby but it seems like it didn't address the core problem - Still getting errors that the self test query is failing because the tables within the metastore do not exist. I took a look around your project, and

Re: Running Hive 3.1.2 embedded in JVM for testing

Hi James, I am doing something similar with the difference that everything runs on docker [1]. I am using Hive 3.1 (HDP though) but things work fine at least with in-memory derby. javax.jdo.option.ConnectionURL jdbc:derby:memory:metastore;create=true Best, Stamatis

Running Hive 3.1.2 embedded in JVM for testing

Hey folks, I have a project where I test with Hive using an embedded HiveServer2 instance within a JVM running integration tests. This has worked for Hive 1.2.2 in the past, and I've been able to get it to work with Hive 2.3.8, but have been having trouble getting it working on Hive 3.0+ The

AW: Migration of Hadoop Warehouse to newer versions lead to bad performance for JDBC-data-retrieval

Hello Mich, Thank you very much for your answers! I will have a look at these points in the morning tomorrow. Kudos for the answers, J Julien Tane Big Data Engineer [Tel.] +49 721 98993-393 [Fax] +49 721 98993-66 [E-Mail]j...@solute.de solute GmbH

Re: Migration of Hadoop Warehouse to newer versions lead to bad performance for JDBC-data-retrieval

Hi Julien, I am not aware that either Hive or HDFS logs provide matrix on performance of either. However, tools like Ganglia should give you some performance matrix. Your Haddop administrator ideally should already have such tools somewhere and through Hortonworks Data

AW: Migration of Hadoop Warehouse to newer versions lead to bad performance for JDBC-data-retrieval

Hello Mitch, Hello all, First of all. Thanks to you. we appreciate your input, yet we would hope for more specific hints or details on how to thoroughly evaluate the speed of the retrieval. The disk performance on the machine was already a good step. yes you understood well. but the newer

Re: Migration of Hadoop Warehouse to newer versions lead to bad performance for JDBC-data-retrieval

Hi Juuien, I assume you mean you are using JDBC drivers to retrieve from the source table in Hive (older version) to the target table in Hive (newer version). 1) what JDBC drivers are you using? 2) Are these environments kerberized in both cases? 3) Have you considered other JDBC drivers for

Migration of Hadoop Warehouse to newer versions lead to bad performance for JDBC-data-retrieval

Dear all, we are in the process of switching from our old cluster with HDP 2.5: HDFS2.7.3 YARN2.7.3 Tez 0.7.0 Hive1.2.1000 to a new cluster with HDP 3.1: HDFS3.1.1.3.1 YARN3.1.0 HIVE3.0.0.3.1 Tez 0.9.0.3.1 We (1st) query and (2nd) retrieve data from table_0

slow create external table on s3

Hello, I was running some create partitioned external table queries looking like: # 30 partitions inside CREATE EXTERNAL TABLE table1 (value string) PARTITIONED BY (shard string) LOCATION 's3a://path/date=2021-02-01/'; INFO : Completed compiling command(queryId=); Time taken: 7.753 seconds # 60

Ability to pass client metadata while calling hive metastore

Hi Hive Users, I am looking for a way to pass some client metadata (like: user agent in http world) while calling hive metastore thrift API in order to be able to trace who and in what context is making the call. Example is I would like to know "add_partition" for table X being called by airflow

hacked page

https://cwiki.apache.org/confluence/display/Hive/Home#Home-HiveVersionsandBranchesHiveVersionsHiveVersionsandBranches In this Page, the link of [Full-Text Search over All Hive Resources] is hacked? can anyone handle this link? [image: 20210207100036.jpg]

Does Remote MetaStore Server support Ranger authorization?

Hello?? Does Remote MetaStore Server support Ranger authorization? Thank you

Does Remote MetaStore Server support Ranger authorization?

Hello?? Does Remote MetaStore Server support Ranger authorization? Thank you

Re: Hive3 LLAP without Slider

Thanks, Panos. Will check Thanks & Regards Amithsha On Thu, Jan 21, 2021 at 10:47 PM Panos Garefalakis wrote: > Hello there, > > Apache Slider has been retired in favor of the YARN Service framework for > a while now. > I believe HIVE-18037 addressed this change on the Hive side -- for >

Re: Hive3 LLAP without Slider

Hello there, Apache Slider has been retired in favor of the YARN Service framework for a while now. I believe HIVE-18037 addressed this change on the Hive side -- for conf/deployment details this post might be useful as well. Cheers,

Hive3 LLAP without Slider

Team, Is that possible to deploy LLAP without a slider? I am using apache Hadoop so want to deploy the LLAP on hive servers without a slider because the project has been retired. I couldn't find any doc related to the deployment of LLAP on Hive. Thanks & Regards Amithsha

[HIVE V1.1.0] Obtaining the query plan using the Java API

Hey folks! Currently we're running Hive 1.1.0 on prem, with the inability to upgrade it easily. My team has been tasked with the problem of obtaining column level lineage information, and being able to map the flow of data through our environment. The current proposal is to establish a

[ANNOUNCE] Apache Hive 2.3.8 Released

The Apache Hive team is proud to announce the release of Apache Hive version 2.3.8. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides, among others: * Tools to enable easy

Override hive.metastore.disallow.incompatible.col.type.changes

Hi I'm struggling to override the 'hive.metastore.disallow.incompatible.col.type.changes' conf. I've got a table (Parquet format) which needs some columns renamed/dropped, structs changed, Hive cli doesn't have the option to drop columns so I'm going Hive thrift api route but keep getting the

Re: Upgrade ORC to 1.6

Hey Laszlo, Thanks for taking a look! Yes, releasing ORC-1.6.7 is definitely the way to go (snapshot was only used for testing) and I am planning to start a discussion on the ORC list soon. However, I wanted to get feedback from the Hive community first and address any potential concerns for

Re: Upgrade ORC to 1.6

Hi! Thanks, Panos, that's very cool! I guess we won't depend on a snapshot, is there a chance that ORC community can fix issues in the near future and release 1.6.7 officially to depend upon? Regards, Laszlo Bodor Panos Garefalakis ezt írta (időpont: 2021. jan. 14., Cs, 18:33): > Hello Hive

Upgrade ORC to 1.6

Hello Hive team, I am happy to announce that as of today Apache HIve precommit tests with ORC-1.6.7 (snapshot) are passing on master branch! There were more than a few compatibility and bug fixes required to make the jump from 1.5 but the version upgrade actually enables (among other things):

Re: Is Insert Overwrite table partition on s3 is an atomic operation ?

Hi Mark It’s my understanding that when you do an INSERT OVERWRITE into a partition, Hive will take out an exclusive lock on the partition and a shared lock on the table itself. This blocks are read and write operations on the partition, and allows reads against the other partitions to

Re: Is Insert Overwrite table partition on s3 is an atomic operation ?

Hi Mark, By atomic operation I gather you mean INSERT/OVERWRITE affects that partition only? According to my somehow dated scripts yes you can do that. The idea being that you only want to overwrite data for that partition ONLY. --show create table marketData; --Populate target table select

Is Insert Overwrite table partition on s3 is an atomic operation ?

Hello Hive users, We are using AWS Glue as Hive compatible metastore when running queries on EMR. For Hive external tables we are using AWS S3. After looking at the docs we didn't find a conclusive answer on whether an Insert Overwrite table partition is an atomic operation, maybe we've missed

mm tables with original files in hive 4

Hi I have a question about how to reconcile the following hive two features going forward: * original files in MM tables (HIVE-19258, fixed in hive 3.1.0, 4.0.0) * the separation of transactional tables from non transactional tables (HIVE-22342, HIVE-22189, fixed in hive 4.0.0). If I

Requesting Write access to the Hive Wiki: Username - achennagiri

Partition optimization problem

I now have the following statement: - tbl1 table DDL: CREATE TABLE tbl1( col1 string, col2 string, col3 string ) PARTITIONED BY (col4 string) -- The partition filed is a date in -MM-dd format stored as orc;

RE: user Digest 19 Jul 2020 19:53:41 -0000 Issue 3895

unsubsribe

RE: user Digest 19 Jul 2020 19:53:41 -0000 Issue 3895

unsubsribe -Original Message- From: user-digest-h...@hive.apache.org Sent: Sunday, July 19, 2020 12:54 PM To: user@hive.apache.org Subject: user Digest 19 Jul 2020 19:53:41 - Issue 3895 user Digest 19 Jul 2020 19:53:41 - Issue 3895 Topics (messages 27202 through 27202) MR3

Re: What are the new features of Hive3?

Hi, Found a nice slide deck that explains the new features in Hive 3.0. - https://www.slideshare.net/Hadoop_Summit/what-is-new-in-apache-hive-30 You might like these links also, 1.

What are the new features of Hive3?

Hello?? What are the new features of Hive3? Thank you

Hive Custom Simple Edge NullPointerException

Hi. I'm using Hive 3.1.0 (Tez Execution Engine) and I'm running into this NPE: INFO : Dag name: WITH event_agg AS (WITH outcome AS (SE...ASC (Stage-1) ERROR : Failed to execute tez graph. java.lang.NullPointerException: null at

What is the role of "hive.metastore.execute.setugi"?

Hello?? What is the role of "hive.metastore.execute.setugi"? Thank you

Re: KDC can't fufill requested option while renewing credentials

Hi, Wondering if https://community.cloudera.com/t5/Support-Questions/WARN-security-UserGroupInformation-Exception-encountered/td-p/123970 is the same issue as yours. Narayanan On Fri, Nov 13, 2020 at 6:44 PM qq <987626...@qq.com> wrote: > Hello： > > The Hive MetaStore log shows the following

KDC can't fufill requested option while renewing credentials

Hello?? The Hive MetaStore log shows the following error: 2020-11-13T11:11:41,014 WARN [TGT Renewer for metastore/h...@test.com] security.UserGroupInformation: Exception encountered while running the renewal command for metastore/h...@test.com. (TGT end time:1605237821000, renewalFailures:

KDC can't fufill requested option while renewing credentials

Hello The Hive MetaStore log shows the following error: 2020-11-13T11:11:41,014 WARN [TGT Renewer for metastore/h...@test.com] security.UserGroupInformation: Exception encountered while running the renewal command for metastore/h...@test.com. (TGT end time:1605237821000, renewalFailures:

Re: How useful are tools for Hive data modeling

Hey Mich, I agree with Austin's reply, a fundamental way of skipping data reading that is not necessary for the query is table partitioning so that would be the first thing to check (along with skewness). Columnar formats such as Parquet, and ORC come with row group statistics (such as min/max

Re: How useful are tools for Hive data modeling

Hi Mich Understood, I was thinking along the lines of the tool being able to auto-generate SQL join syntax etc, rather than in terms of scan performance. I’m not so familiar with Parquet with Hive. I know that Parquet also has min and max indexes, and more recently bloom filters. However, I

Re: How useful are tools for Hive data modeling

Many thanks Austin. The challenge I have been told is how to effectively query a subset of data avoiding full table scan. The tables I believe are parquet. I know performance in Hive is not that great, so anything that could help would be great. Cheers, LinkedIn *

Re: How useful are tools for Hive data modeling

Hi Mich Hive also has non-validated primary key, foreign key etc constraints. Whilst I’m not too familiar with the modelling tools you mention, perhaps they’re able to use these for generating SQL etc? ORC files have indexes (min, max, bloom filters) - not particularly relevant to the data

Re: How useful are tools for Hive data modeling

Many thanks Peter. LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or

Re: How useful are tools for Hive data modeling

Hi Mich, Index support was removed from hive: https://issues.apache.org/jira/browse/HIVE-21968 https://issues.apache.org/jira/browse/HIVE-18715 Thanks, Peter > On Nov 11, 2020, at 17:25, Mich

Fwd: How useful are tools for Hive data modeling

Hi all, I wrote these notes earlier this year. I heard today that someone mentioned Hive 1 does not support indexes but hive 2 does. I still believe that Hive does not support indexing as per below. Has this been changed? Regards, Mich -- Forwarded message - From: Mich

What is the difference between varname and hivename in MetastoreConf.java?

What is the difference between varname and hivename in the picture below?

How is the Hive MetaStore authenticated for accessing HDFS?

Hey, How is the Hive MetaStore authenticated for accessing HDFS? Will it be automatically re-certified when the certification expires? Is it related to hive.metastore.kerberos.key.file and hive.metastore.kerberos.principal? Thank you

Re: what does MM stand for

MM -> Micro Managed Hive managed tables supporting Insert-only operations with ACID semantics are called MM (Micro-Managed) OR Insert-Only ACID tables. Supports all file formats. >From say

what does MM stand for

Hello Apologies for the silly question. What you say "MM tables", what does MM stand for? HIVE_MM_ALLOW_ORIGINALS("hive.mm.allow.originals", false,     "Whether to allow original files in MM tables. Conversion to MM may be expensive if\n" +     "this is set to false, however unless

Does Hive need to rely on Hadoop to deploy remote MetStore?

Hey, Does Hive need to rely on Hadoop to deploy remote MetStore? Thank you,

Re: SQL CTAS query failed on compilation stage

ok fine. LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of

Re: SQL CTAS query failed on compilation stage

I understand, but it looks strange as a query without the "create table" clause works. Obviously I use the workaround, but I think Hive as an application should be more predictable in interaction, so I created a JIRA for the issue. https://issues.apache.org/jira/browse/HIVE-24352 Regards,

Re: SQL CTAS query failed on compilation stage

well you have to be pragmatic. That may well be a bug due to Hive, especially it says "Also check for circular dependencies" you can raise a JIRA but not sure about its priority as you have a work-around HTH LinkedIn *

Re: SQL CTAS query failed on compilation stage

Hi Mich, Thank you for the reply! Creating a stage table works well, a problem comes up when CTE or subquery in from clause is used. wt., 3 lis 2020 o 10:45 Mich Talebzadeh napisał(a): > Hm, > > Hi Bartosz, > > Can you create a temporary table with your sub-query and see it works? > > create

Re: SQL CTAS query failed on compilation stage

Hm, Hi Bartosz, Can you create a temporary table with your sub-query and see it works? create temporary table tab2 as ... HTH LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

SQL CTAS query failed on compilation stage

Hi! I use Hive 3.1.0 and beeline. I have encountered a compilation error when issue a CTAS query from beeline, but without "create table" query works as expected,* narrowed query to reproduce:* create table tab_error as with tab2 as ( select id, lead(id) over (partition by id

Re: Hive SQL extension

Hi Jesus, Stamatis Thanks for taking the time and answering my questions! Here is some info I have started from: Iceberg partitioning is defined here: https://iceberg.apache.org/spec/#partitioning Other examples for SQL extension proposal from

Re: Hive Avro: Directly use of embedded Avro Scheme

Hey Dennis, Specifying the schema url is simply a convenience tool so you can have a single schema defined instead of having a SQL schema (CREATE TABLE) and a separate Avro schema file which reduces maintenance overhead and prevents a situation where the two could potentially fall out of sync.

AW: Hive Avro: Directly use of embedded Avro Scheme

Understood. So to hold the schema stable you should have an external reference to an avrc url (eg registry) which can evolve. And checking new Avro against registry is made easy because avrc is embedded. And if changed you can easily create a new version. Is this the idea ? Br, Dennis

Re: Hive Avro: Directly use of embedded Avro Scheme

What would your expectation be? That Hive reads the first file it finds and uses that schema in the table definition? What if the table is empty and a user attempts an INSERT? What should be the behavior? The real power of Avro is not so much that the schema can exist (optionally) in the file

Hive Avro: Directly use of embedded Avro Scheme

Hello Support, currently I have created the following AVRO Hive table which works fine. CREATE EXTERNAL TABLE blahblah.blublub STORED AS AVRO LOCATION "/***/in" TBLPROPERTIES ('avro.schema.url‘=‚/.../schema/blublub.avsc') As you can see I need to use the schema 'avro.schema.url' property

Re: org.apache.thrift.transport.TTransportException:Invalid status -128

Hello, Your stack says that the client-facing principal is not set However it goes on to say that the server side principal is set and then reports a successful login The following links give details instructions on how to configure the hive server with kerberos and how to configure a JDBC

?????? org.apache.thrift.transport.TTransportException:Invalid status -128

When Hive MetaStore enables Kerberos authentication, what configuration needs to be set in the Client? ---- ??: "dev"

?????? How to set the Kerberos mode of HIve MetaStore?

thank you very much, ---- ??: "user"

MR3 1.2 released

Hello Hive users, MR3 1.2 has been released. A few improvements in this release are: 1. MR3 can publish Prometheus metrics. 2. On Kubernetes, the user can change the total resources for workers dynamically (e.g., by using Prometheus metrics). This feature can be combined with autoscaling in

Re: How to set the Kerberos mode of HIve MetaStore?

Hi, I found the following documentation in this link https://docs.cloudera.com/documentation/enterprise/5-12-x/topics/cdh_sg_hiveserver2_security.html#topic_9_1 The Hive metastore server supports Kerberos authentication for Thrift clients. For example, you can configure a standalone Hive

How to set the Kerberos mode of HIve MetaStore?

hi! How to set the Kerberos mode of HIve MetaStore?

Re: Hive SQL extension

Hi Peter, Thanks for bringing this up. Why are targeting the 'partition by spec' syntax? Is it for convenience? Was it already introduced by Iceberg? I did not understand the reasoning for not introducing the new syntax in Hive. As it was already mentioned by Stamatis, there is some advantage

Re: Hive SQL extension

I do like extensions and things that simplify our life when writing queries. Regarding the partitioning syntax for Iceberg, there may be better alternatives. I was also leaning towards a syntax like the one proposed by Jesus (in another thread) based on virtual columns, which is also part of SQL

SMB joins not working on partitioned queries

Hi all, I'm trying to understand (for the last months) why SMB joins (the most efficient ones) are unable to work with partitions. All my being tells me it should work but, it doesn't. I would really, really, (really!) appreciate any insights on this problem. First, the working example without

Re: Hive SQL extension

Hi all, I do not know if that may be of interest to you, but there are other projects that could benefit from this. For instance, ADQL (Astronomical Data Query Language) is a SQL-like language that defines some higher-level

Re: Hive SQL extension

Let's assume that this feature would be useful for Iceberg tables, but useless and even problematic/forbidden for other tables. :) My thinking is, that it could make Hive much more user friendly, if we would allow for extensions in language. With Iceberg integration we plan to do several

Re: Hive SQL extension

Hi Peter, I am nowhere near being an expert but just wanted to share my thoughts. If I understand correctly you would like some syntactic sugar in Hive to support partitioning as per Iceberg. I cannot tell if that's really useful or not but from my point of view it doesn't seem a very good idea

Hive SQL extension

Hi Hive experts, I would like to extend Hive SQL language to provide a way to create Iceberg partitioned tables like this: create table iceberg_test( level string, event_time timestamp, message string, register_time date, telephone array )

<    3   4   5   6   7   8   9   10   11   12   >