[jira] [Commented] (HIVE-26473) Upgrade to Java17
[ https://issues.apache.org/jira/browse/HIVE-26473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580072#comment-17580072 ] Sylwester Lachiewicz commented on HIVE-26473: - To easier track, please add links to above project's jira tickets. Thx > Upgrade to Java17 > - > > Key: HIVE-26473 > URL: https://issues.apache.org/jira/browse/HIVE-26473 > Project: Hive > Issue Type: Improvement >Reporter: dingwei2019 >Priority: Major > > we know that jdk11 is a LTS version, but the technical support will be end in > September 2023. JDK17 is the next generation LTS version, and will support a > least to 2026. > for G1GC, Java17 will get 8.66% faster than Java11, for ParallelGC, the > percent will be 6.54%. If we upgrade to java17, we will get more performance > improvementthan Java11. > > I suggest, we upgrade hive version to support java17. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26468) Cleanup dependency to FlatBuffers
[ https://issues.apache.org/jira/browse/HIVE-26468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26468: Description: Remove issue with overlapping dependencies to FlatBuffers. Warning inside shading step for hive-jdbc, hive-beeline shows that we have wring dependencies. Similar issue exists inside hive-serde. Simply switch to Google FlatBuffers artefact [WARNING] flatbuffers-java-1.9.0.jar, flatbuffers-java-1.6.0.1.jar define 5 overlapping classes: [WARNING] - com.google.flatbuffers.Struct [WARNING] - com.google.flatbuffers.Table$1 [WARNING] - com.google.flatbuffers.Constants [WARNING] - com.google.flatbuffers.Table [WARNING] - com.google.flatbuffers.FlatBufferBuilder was: Remove issue with overlapping dependencies to FlatBuffers. Warning inside shading step for hive-jdbc shows that we have wring dependencies. Similar issue exists inside hive-serde. Simply switch to Google FlatBuffers artefact [WARNING] flatbuffers-java-1.9.0.jar, flatbuffers-java-1.6.0.1.jar define 5 overlapping classes: [WARNING] - com.google.flatbuffers.Struct [WARNING] - com.google.flatbuffers.Table$1 [WARNING] - com.google.flatbuffers.Constants [WARNING] - com.google.flatbuffers.Table [WARNING] - com.google.flatbuffers.FlatBufferBuilder > Cleanup dependency to FlatBuffers > - > > Key: HIVE-26468 > URL: https://issues.apache.org/jira/browse/HIVE-26468 > Project: Hive > Issue Type: Bug >Reporter: Sylwester Lachiewicz >Priority: Minor > > Remove issue with overlapping dependencies to FlatBuffers. > Warning inside shading step for hive-jdbc, hive-beeline shows that we have > wring dependencies. Similar issue exists inside hive-serde. > > Simply switch to Google FlatBuffers artefact > > [WARNING] flatbuffers-java-1.9.0.jar, flatbuffers-java-1.6.0.1.jar define 5 > overlapping classes: > [WARNING] - com.google.flatbuffers.Struct > [WARNING] - com.google.flatbuffers.Table$1 > [WARNING] - com.google.flatbuffers.Constants > [WARNING] - com.google.flatbuffers.Table > [WARNING] - com.google.flatbuffers.FlatBufferBuilder -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26331) Use maven-surefire-plugin version consistently in standalone-metastore modules
[
https://issues.apache.org/jira/browse/HIVE-26331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17554578#comment-17554578
]
Sylwester Lachiewicz commented on HIVE-26331:
-
found same issue while working on preparing to update ASF HIVE-26063
maybe we can create hive-parent-pom as common to all our parts - so later we
can centrally manage plugins/parent poms and part of dependences ?
> Use maven-surefire-plugin version consistently in standalone-metastore modules
> --
>
> Key: HIVE-26331
> URL: https://issues.apache.org/jira/browse/HIVE-26331
> Project: Hive
> Issue Type: Task
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Due to some problems in the pom.xml files inside the standalone-metastore
> modules we end up using different maven-surefire-plugin versions.
> Most of the modules use 3.0.0-M4, which is the expected one, while
> the {{hive-standalone-metastore-common}} uses the older 2.22.0 version.
> +Actual+
> {noformat}
> [INFO] --- maven-surefire-plugin:2.22.0:test (default-test) @
> hive-standalone-metastore-common ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> hive-metastore ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> hive-standalone-metastore-server ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> metastore-tools-common ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> hive-metastore-benchmarks ---
> {noformat}
> The goal of this JIRA is to ensure we use the same version consistently in
> all modules.
> +Expected+
> {noformat}
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> hive-standalone-metastore-common ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> hive-metastore ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> hive-standalone-metastore-server ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> metastore-tools-common ---
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @
> hive-metastore-benchmarks ---
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HIVE-25635) Upgrade Thrift to 0.16.0
[ https://issues.apache.org/jira/browse/HIVE-25635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-25635: Summary: Upgrade Thrift to 0.16.0 (was: Upgrade Thrift to 0.15.0) > Upgrade Thrift to 0.16.0 > > > Key: HIVE-25635 > URL: https://issues.apache.org/jira/browse/HIVE-25635 > Project: Hive > Issue Type: Improvement >Reporter: Yuming Wang >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > To addresses CVEs: > ||Component Name||Component Version Name||Vulnerability||Fixed version|| > |Apache > Thrift|0.11.0-4.|[CVE-2020-13949|https://github.com/advisories/GHSA-g2fg-mr77-6vrm]|0.14.1| -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-24483) Bump protobuf version to 3.21.1 (aarch64 support)
[ https://issues.apache.org/jira/browse/HIVE-24483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-24483: Summary: Bump protobuf version to 3.21.1 (aarch64 support) (was: Bump protobuf version to 3.12.0 (aarch64 support)) > Bump protobuf version to 3.21.1 (aarch64 support) > - > > Key: HIVE-24483 > URL: https://issues.apache.org/jira/browse/HIVE-24483 > Project: Hive > Issue Type: Improvement >Reporter: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-21588) Remove HBase dependency from hive-metastore
[ https://issues.apache.org/jira/browse/HIVE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-21588: Resolution: Fixed Status: Resolved (was: Patch Available) > Remove HBase dependency from hive-metastore > --- > > Key: HIVE-21588 > URL: https://issues.apache.org/jira/browse/HIVE-21588 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 4.0.0 >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21588.01.patch, HIVE-21588.02.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > HIVE-17234 has removed HBase metastore from master. But maven dependency have > not been removed. We should remove it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-21588) Remove HBase dependency from hive-metastore
[ https://issues.apache.org/jira/browse/HIVE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-21588: Fix Version/s: 4.0.0 > Remove HBase dependency from hive-metastore > --- > > Key: HIVE-21588 > URL: https://issues.apache.org/jira/browse/HIVE-21588 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 4.0.0 >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21588.01.patch, HIVE-21588.02.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > HIVE-17234 has removed HBase metastore from master. But maven dependency have > not been removed. We should remove it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-25156) reduce数设置为2后，查出的数据减少一半
[ https://issues.apache.org/jira/browse/HIVE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25156. - Resolution: Auto Closed > reduce数设置为2后，查出的数据减少一半 > -- > > Key: HIVE-25156 > URL: https://issues.apache.org/jira/browse/HIVE-25156 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2 >Reporter: kang >Priority: Major > > c'c我的hive版本为3.1.2，当我设置reduce数后，返回的结果会是 结果数/reduce数 的数据量。 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-25262) Hive lineage dir does not have any log？
[ https://issues.apache.org/jira/browse/HIVE-25262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25262. - Resolution: Auto Closed > Hive lineage dir does not have any log？ > --- > > Key: HIVE-25262 > URL: https://issues.apache.org/jira/browse/HIVE-25262 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.1 >Reporter: roamer_wu >Priority: Minor > > I am using hive2.1.1 with CDH6.3.1. Hive Property `Enable Lineage Collection` > is default Enabled, but the log Dir `/var/log/hive/lineage` is empty. > [cdh hive > config|https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cn_admcfg_impala_lineage_log.html] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-25647) hadoop memo
[
https://issues.apache.org/jira/browse/HIVE-25647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylwester Lachiewicz resolved HIVE-25647.
-
Resolution: Invalid
> hadoop memo
> ---
>
> Key: HIVE-25647
> URL: https://issues.apache.org/jira/browse/HIVE-25647
> Project: Hive
> Issue Type: Wish
> Components: Configuration
>Affects Versions: 3.1.2
> Environment: hadoop 2.7.3
>Reporter: St Li
>Assignee: St Li
>Priority: Major
> Fix For: All Versions
>
> Attachments: worldip.csv
>
>
> master,slave1,slave2master,slave1,slave2//opt represent wechat hadoop bigdata
> dev//2019 :bigdata competitionhadoop 50070hbase 16010storm 8080
> #hostnamehostnamectl set-hostname master && bash hostname master && bash
> hostname slave1/slave2 && bash vim /etc/hostname master/slave1/slave2vim
> /etc/hosts ip master ip slave1 ipslave2
> #yumcd /etc/yum.repos.d && rm -rf *wget
> http://172.16.47.240/bigdata/repofile/bigdata.repoyum clean all
> #firewallsystemctl stop firewalldsystemctl status firewalld
> #timezonetzselect 5-9-1-1echo "TZ='Asia/Shanghai'; export TZ" >>
> /etc/profile && source /etc/profile
> #ntpyum install -y ntpvim /etc/ntp.conf//#server 0~3.centos.pool.ntp.org
> iburstserver 127.127.1.0fudge 127.127.1.0 stratum 10/bin/systemctl restart
> ntpd.servicentpdate master (slave1,slave2)
> #crontabservice crond status/sbin/service crond startcrontab -e*/30 8-17 * *
> * /usr/sbin/ntpdate mastercrontab –l
> #ssh passwordssh-keygen -t dsa -P '' -f ~/.ssh/id_dsacat
> /root/.ssh/id_dsa.pub >> /root/.ssh/authorized_keysscp ~/.ssh/authorized_keys
> root@slave1:~/.ssh/scp ~/.ssh/authorized_keys root@slave2:~/.ssh/
> ssh-copy-id masterssh-copy-id slave1ssh-copy-id slave2
> #install jdkmkdir -p /usr/javatar -zxvf jdk-8u171-linux-x64.tar.gz -C
> /usr/java/
> vim /etc/profileexport JAVA_HOME=/usr/java/jdk1.8.0_171export
> CLASSPATH=$JAVA_HOME/lib/export PATH=$PATH:$JAVA_HOME/bin
> source /etc/profile && java -version
> scp -r /usr/java root@slave1:/usr/scp -r /usr/java root@slave2:/usr/
> #install hadoopmkdir -p /usr/hadoop && cd /usr/hadooptar -zxvf
> /usr/hadoop/hadoop-2.7.3.tar.gz -C /usr/hadoop/rm -rf
> /usr/hadoop/hadoop-2.7.3.tar.gzvim /etc/profileexport
> HADOOP_HOME=/usr/hadoop/hadoop-2.7.3export
> PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbinhadoop //test
> hadoop-env.sh/mapred-env.sh/yarn-en.shexport JAVA_HOME=/usr/java/jdk1.8.0_171
> ##vim core-site.xmlfs.default.name \{hdfs://master:9000}hadoop.tmp.dir
> \{/usr/hadoop/hadoop-2.7.3/hdfs/tmp}io.file.buffer.size
> \{131072}fs.checkpoint.period \{60}fs.checkpoint.size \{67108864}
> ##hdfs-site.xmldfs.replication \{2}dfs.namenode.name.dir
> \{file:/usr/hadoop/hadoop-2.7.3/hdfs/name}dfs.datanode.data.dir
> \{file:/usr/hadoop/hadoop-2.7.3/hdfs/data}
> ##vim yarn-env.shyarn.resourcemanager.address
> \{master:18040}yarn.resourcemanager.scheduler.address
> \{master:18030}yarn.resourcemanager.webapp.address
> \{master:18088}yarn.resourcemanager.resource-tracker.address
> \{18025}yarn.resourcemanager.admin.address
> \{master:18141}yarn.nodemanager.aux-services
> \{mapreduce_shuffle}yarn.nodemanager.auxservices.mapreduce.shuffle.class
> \{org.apache.hadoop.mapred.ShuffleHandler}
> #vim mapred-site.xmlmapreduce.framework.name \{yarn}
> #slaves fileecho master > master && echo slave1 > slaves && echo slave2 >>
> slaves
> #hadoop formathadoop namenode -format (master) //has been
> successfully#start hadoopstart-all.shmaster
> :NameNode,SecondaryNameNode,ResourceManagerslave1~2:DataNode,NodeManager
> start-dfs.shstart-yarn.shhadoop-daemon.sh start namenodehadoop-daemon.sh
> start datanodehadoop-daemon.sh start secondarynamenodehadoop-daemon.sh start
> resourcemanagerhadoop-daemon.sh start nodemanager
> test hdfs& mapreducehadoop fs -mkdir /inputhadoop fs -put
> $HADOOP_HOME/README.txt /input
> http://master:50070hadoop jar
> $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.4.jar
> #install hiveyum -y install mysql-community-server slave2:mysqlserver
> slave1:hiveserver master:hiveclientsystemctl daemon-reloadsystemctl start
> mysqldcat /var/log/mysqld.log grep "temporary password"mysql -uroot -pset
> global validate_password_policy=0;set global validate_password_length=4;alter
> user 'root'@'localhost' identified by '123456';mysql -uroot -p123456create
> user 'root'@'%' identified by '123456';grant all privileges on *.* to
> 'root'@'%' with grant option;flush privileges;
> mkdir -p /usr/hive tar -zxvf /usr/hive/apache-hive-2.1.1-bin.tar.gz -C
> /usr/hive/
> vim /etc/profile //for hiveexport
> HIVE_HOME=/usr/hive/apache-hive-2.1.1-binexport
> PATH=$PATH:$HIVE_HOME/binsource /etc/profile
> vim hive-env.shcd $HIVE_HOME/conf && vim hive-env.shexport
> HADOOP_HOME=/usr/hadoop/hadoop-2.7.3export
>
[jira] [Resolved] (HIVE-25496) hadoop 3.3.1 / hive 3.2.1 / OpenJDK11 compatible?
[ https://issues.apache.org/jira/browse/HIVE-25496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25496. - Resolution: Duplicate > hadoop 3.3.1 / hive 3.2.1 / OpenJDK11 compatible? > - > > Key: HIVE-25496 > URL: https://issues.apache.org/jira/browse/HIVE-25496 > Project: Hive > Issue Type: Bug > Environment: Linux VM >Reporter: Jerome Le Ray >Assignee: Jerome Le Ray >Priority: Major > > We used the following configuration > hadoop 3.2.1 > hive 3.1.2 > PostGres 12 > Java - OracleJDK 8 > For internal reasons, we have to migrate to OpenJDK11. > So, I've migrated hadoop 3.2.1 to the new version hadoop 3.3.1 > When I'm starting the hiveserver2 service, I've got the error : > which: no hbase in > (/usr/local/bin:/bin:/usr/pgsql-12/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/jdk-11.0.10+9/bin:/opt/hivemetastore/hadoop-3.3.1/bin:/opt/hivemetastore/apache-hive-3.1.2-bin/b > in) > 2021-09-02 16:48:05: Starting HiveServer2 > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/opt/hivemetastore/hadoop-3.3.1/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/opt/hivemetastore/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > 2021-09-02 16:48:06,744 INFO conf.HiveConf: Found configuration file > file:/opt/hivemetastore/apache-hive-3.1.2-bin/conf/hive-site.xml > 2021-09-02 16:48:07,169 WARN conf.HiveConf: HiveConf of name > hive.metastore.local does not exist > 2021-09-02 16:48:07,169 WARN conf.HiveConf: HiveConf of name > hive.metastore.thrift.bind.host does not exist > 2021-09-02 16:48:07,170 WARN conf.HiveConf: HiveConf of name > hive.enforce.bucketing does not exist > 2021-09-02 16:48:08,414 INFO server.HiveServer2: STARTUP_MSG: > / > STARTUP_MSG: Starting HiveServer2 > STARTUP_MSG: host = lhroelcspt1001.enterprisenet.org/10.90.122.159 > STARTUP_MSG: args = [-hiveconf, mapred.job.tracker=local, -hiveconf, > fs.default.name=file:///cip-data, -hiveconf, > hive.metastore.warehouse.dir=file:cip-data, --hiveconf, hive.server2.thrif > t.port=1, --hiveconf, hive.root.logger=INFO,console] > STARTUP_MSG: version = 3.1.2 > (...) > STARTUP_MSG: build = git://HW13934/Users/gates/tmp/hive-branch-3.1/hive -r > 8190d2be7b7165effa62bd21b7d60ef81fb0e4af; compiled by 'gates' on Thu Aug 22 > 15:01:18 PDT 2019 > / > 2021-09-02 16:48:08,436 INFO server.HiveServer2: Starting HiveServer2 > 2021-09-02 16:48:08,462 WARN conf.HiveConf: HiveConf of name > hive.metastore.local does not exist > 2021-09-02 16:48:08,463 WARN conf.HiveConf: HiveConf of name > hive.metastore.thrift.bind.host does not exist > 2021-09-02 16:48:08,463 WARN conf.HiveConf: HiveConf of name > hive.enforce.bucketing does not exist > Hive Session ID = 440449ff-99b7-429c-82d9-e20bdcc9b46f > 2021-09-02 16:48:08,566 INFO SessionState: Hive Session ID = > 440449ff-99b7-429c-82d9-e20bdcc9b46f > 2021-09-02 16:48:08,566 INFO server.HiveServer2: Shutting down HiveServer2 > 2021-09-02 16:48:08,584 INFO server.HiveServer2: Stopping/Disconnecting tez > sessions. > 2021-09-02 16:48:08,585 WARN server.HiveServer2: Error starting HiveServer2 > on attempt 1, will retry in 6ms > java.lang.RuntimeException: Error applying authorization policy on hive > configuration: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot > be cast to class java.net.URLClassLoader (jdk. > internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are > in module java.base of loader 'bootstrap') > at org.apache.hive.service.cli.CLIService.init(CLIService.java:118) > at org.apache.hive.service.CompositeService.init(CompositeService.java:59) > at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:230) > at > org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1036) > at > org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140) > at > org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1305) > at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1149) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >
[jira] [Updated] (HIVE-25597) Bump Apache Derby 10.14.1.0 to 10.14.2.0 CVE-2018-1313
[ https://issues.apache.org/jira/browse/HIVE-25597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-25597: Issue Type: Improvement (was: Bug) > Bump Apache Derby 10.14.1.0 to 10.14.2.0 CVE-2018-1313 > -- > > Key: HIVE-25597 > URL: https://issues.apache.org/jira/browse/HIVE-25597 > Project: Hive > Issue Type: Improvement >Reporter: Ranith Sardar >Assignee: Ranith Sardar >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-25612) Fix flakiness in test in `standalone-metastore`
[
https://issues.apache.org/jira/browse/HIVE-25612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylwester Lachiewicz resolved HIVE-25612.
-
Resolution: Abandoned
> Fix flakiness in test in `standalone-metastore`
> ---
>
> Key: HIVE-25612
> URL: https://issues.apache.org/jira/browse/HIVE-25612
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.1.2
>Reporter: Alexander Ronald Altman
>Priority: Trivial
> Labels: test-stability, utils
>
> {{The test method
> org.apache.hadoop.hive.metastore.utils.TestMetaStoreServerUtils.testGetPartitionspecsGroupedBySDonePartitionCombined
> creates a List that was order-unstable under NonDex
> (https://github.com/TestingResearchIllinois/NonDex); sorting it ensures that
> the partitions are in a known order while preserving the intent of the test.}}
>
> {{See the accompanying GitHub PR link for more.}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (HIVE-25678) java.lang.ClassCastException in COALESCE()
[ https://issues.apache.org/jira/browse/HIVE-25678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25678. - Resolution: Cannot Reproduce > java.lang.ClassCastException in COALESCE() > -- > > Key: HIVE-25678 > URL: https://issues.apache.org/jira/browse/HIVE-25678 > Project: Hive > Issue Type: Improvement >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > > Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable > cannot be cast to org.apache.hadoop.io.BooleanWritable > at > org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableBooleanObjectInspector.get(WritableBooleanObjectInspector.java:36) > at > org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getBoolean(PrimitiveObjectInspectorUtils.java:514) > at > org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$BooleanConverter.convert(PrimitiveObjectInspectorConverter.java:67) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFUtils$ReturnObjectInspectorResolver.convertIfNecessary(GenericUDFUtils.java:247) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFUtils$ReturnObjectInspectorResolver.convertIfNecessary(GenericUDFUtils.java:213) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFWhen.evaluate(GenericUDFWhen.java:105) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:197) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:88) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPEqual.evaluate(GenericUDFOPEqual.java:114) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:197) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:88) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFWhen.evaluate(GenericUDFWhen.java:96) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:197) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:88) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFWhen.evaluate(GenericUDFWhen.java:96) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:197) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:88) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.evaluate(GenericUDFOPAnd.java:65) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:197) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:88) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDFWhen.evaluate(GenericUDFWhen.java:93) > at > org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:197) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorHead._evaluate(ExprNodeEvaluatorHead.java:44) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:68) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-25799) Maven should stop early for unsupported java versions
[ https://issues.apache.org/jira/browse/HIVE-25799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543441#comment-17543441 ] Sylwester Lachiewicz commented on HIVE-25799: - we are working to be able to compile with newer java version > Maven should stop early for unsupported java versions > - > > Key: HIVE-25799 > URL: https://issues.apache.org/jira/browse/HIVE-25799 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure >Reporter: Josh Soref >Assignee: Josh Soref >Priority: Trivial > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > I made the mistake of using my default java compiler (which apparently is 17) > to build hive, and it failed. If hive only supports a certain version (or > ranges of versions) of java, it would be incredibly helpful if maven reported > this eagerly. > This is easily achievable using the maven enforcer system (a number of > enforcers are already deployed in the top level pom): > [https://maven.apache.org/enforcer/enforcer-rules/requireJavaVersion.html] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-25802) Log4j2 Vulnerability in Hive Storage API
[ https://issues.apache.org/jira/browse/HIVE-25802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25802. - Resolution: Fixed > Log4j2 Vulnerability in Hive Storage API > > > Key: HIVE-25802 > URL: https://issues.apache.org/jira/browse/HIVE-25802 > Project: Hive > Issue Type: Bug > Components: storage-api >Affects Versions: 4.0.0 >Reporter: Nikhil Gupta >Priority: Major > Fix For: 4.0.0 > > > Storage API also brings in log4j2 dependency <= 2.14.1 that can still expose > a vulnerability in hive -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-25821) upgrade to log4j 2.17.0 - another CVE: CVE-2021-45105
[ https://issues.apache.org/jira/browse/HIVE-25821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25821. - Fix Version/s: 4.0.0-alpha-1 Resolution: Resolved > upgrade to log4j 2.17.0 - another CVE: CVE-2021-45105 > - > > Key: HIVE-25821 > URL: https://issues.apache.org/jira/browse/HIVE-25821 > Project: Hive > Issue Type: Improvement >Reporter: PJ Fanning >Assignee: BillLucky >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Attachments: HIVE-25821.patch > > Time Spent: 40m > Remaining Estimate: 0h > > https://logging.apache.org/log4j/2.x/security.html -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-24500) Hive - upgrade log4j 2.12.1 to 2.13.2+ due to CVE-2020-9488
[ https://issues.apache.org/jira/browse/HIVE-24500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-24500: Fix Version/s: 4.0.0-alpha-1 > Hive - upgrade log4j 2.12.1 to 2.13.2+ due to CVE-2020-9488 > --- > > Key: HIVE-24500 > URL: https://issues.apache.org/jira/browse/HIVE-24500 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hive is pulling in log4j 2.12.1 specifically to: > * ./usr/lib/hive/lib/log4j-core-2.12.1.jar > CVE-2020-9488 affects this version and the fix is to upgrade to 2.13.2+. So, > upgrade this dependency. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-25984) TTTT
[ https://issues.apache.org/jira/browse/HIVE-25984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25984. - Resolution: Invalid > > > > Key: HIVE-25984 > URL: https://issues.apache.org/jira/browse/HIVE-25984 > Project: Hive > Issue Type: Test >Reporter: lkl >Assignee: lkl >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-26034) [minor] fix grammar error in README
[ https://issues.apache.org/jira/browse/HIVE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-26034. - Resolution: Not A Problem > [minor] fix grammar error in README > --- > > Key: HIVE-26034 > URL: https://issues.apache.org/jira/browse/HIVE-26034 > Project: Hive > Issue Type: Task >Reporter: ZhangJian He >Priority: Minor > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-26164) please delete this
[ https://issues.apache.org/jira/browse/HIVE-26164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-26164. - Resolution: Auto Closed > please delete this > -- > > Key: HIVE-26164 > URL: https://issues.apache.org/jira/browse/HIVE-26164 > Project: Hive > Issue Type: Bug >Reporter: Meylis Matiyev >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26166) Make website GDPR compliant
[ https://issues.apache.org/jira/browse/HIVE-26166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543439#comment-17543439 ] Sylwester Lachiewicz commented on HIVE-26166: - here instructions how to switch to our ASF instance [https://privacy.apache.org/matomo/] > Make website GDPR compliant > --- > > Key: HIVE-26166 > URL: https://issues.apache.org/jira/browse/HIVE-26166 > Project: Hive > Issue Type: Task > Components: Website >Reporter: Stamatis Zampetakis >Priority: Major > > Per the email that was sent out from privacy we need to make the Hive website > GDPR compliant. > # The link to privacy policy needs to be updated from > [https://hive.apache.org/privacy_policy.html] to > [https://privacy.apache.org/policies/privacy-policy-public.html] > # The google analytics service must be removed -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (HIVE-26268) Upgrade snappy-java to 1.1.8.4
[ https://issues.apache.org/jira/browse/HIVE-26268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz reassigned HIVE-26268: --- Assignee: Sylwester Lachiewicz (was: Panagiotis Garefalakis) > Upgrade snappy-java to 1.1.8.4 > -- > > Key: HIVE-26268 > URL: https://issues.apache.org/jira/browse/HIVE-26268 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Sylwester Lachiewicz >Assignee: Sylwester Lachiewicz >Priority: Trivial > Labels: pull-request-available > > Upgrade to get benefits from performance improvements and bug fixes. > Also to support Apple Silicon (M1, Mac-aarch64) > [https://github.com/xerial/snappy-java/blob/master/Milestone.md#snappy-java-1183-2021-01-20] > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-26268) Upgrade snappy-java to 1.1.8.4
[ https://issues.apache.org/jira/browse/HIVE-26268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26268: Description: Upgrade to get benefits from performance improvements and bug fixes. Also to support Apple Silicon (M1, Mac-aarch64) [https://github.com/xerial/snappy-java/blob/master/Milestone.md#snappy-java-1183-2021-01-20] was: Clean transitive dependency mainly coming from hadoop-common, avro and spark-core and also bump version to take advantage of perf improvements, glibc compatibility etc. https://github.com/xerial/snappy-java/blob/master/Milestone.md#snappy-java-117-2017-11-30 > Upgrade snappy-java to 1.1.8.4 > -- > > Key: HIVE-26268 > URL: https://issues.apache.org/jira/browse/HIVE-26268 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Sylwester Lachiewicz >Assignee: Panagiotis Garefalakis >Priority: Trivial > Labels: pull-request-available > > Upgrade to get benefits from performance improvements and bug fixes. > Also to support Apple Silicon (M1, Mac-aarch64) > [https://github.com/xerial/snappy-java/blob/master/Milestone.md#snappy-java-1183-2021-01-20] > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (HIVE-26268) Upgrade snappy-java to 1.1.8.4
[ https://issues.apache.org/jira/browse/HIVE-26268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz reassigned HIVE-26268: --- > Upgrade snappy-java to 1.1.8.4 > -- > > Key: HIVE-26268 > URL: https://issues.apache.org/jira/browse/HIVE-26268 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Sylwester Lachiewicz >Assignee: Panagiotis Garefalakis >Priority: Trivial > Labels: pull-request-available > > Clean transitive dependency mainly coming from hadoop-common, avro and > spark-core and also bump version to take advantage of perf improvements, > glibc compatibility etc. > https://github.com/xerial/snappy-java/blob/master/Milestone.md#snappy-java-117-2017-11-30 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26084) Oracle metastore init tests are flaky
[
https://issues.apache.org/jira/browse/HIVE-26084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541668#comment-17541668
]
Sylwester Lachiewicz commented on HIVE-26084:
-
We may try to use slim images from here
https://hub.docker.com/r/gvenzl/oracle-xe
> Oracle metastore init tests are flaky
> -
>
> Key: HIVE-26084
> URL: https://issues.apache.org/jira/browse/HIVE-26084
> Project: Hive
> Issue Type: Test
> Components: Testing Infrastructure
>Affects Versions: 4.0.0-alpha-2
>Reporter: Peter Vary
>Assignee: Stamatis Zampetakis
>Priority: Major
>
> After HIVE-26022 we started to run the oracle metastore init tests, but they
> seem to be flaky.
> I see this issue quite often:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3147/1/pipeline/551
> We might have to increase the timeout, or use another oracle image for more
> consistent tests.
> The error in the logs for future reference
> {code}
> [2022-03-28T14:10:07.804Z] + echo 127.0.0.1 dev_oracle
> [2022-03-28T14:10:07.804Z] + sudo tee -a /etc/hosts
> [2022-03-28T14:10:07.804Z] 127.0.0.1 dev_oracle
> [2022-03-28T14:10:07.804Z] + . /etc/profile.d/confs.sh
> [2022-03-28T14:10:07.804Z] ++ export MAVEN_OPTS=-Xmx2g
> [2022-03-28T14:10:07.804Z] ++ MAVEN_OPTS=-Xmx2g
> [2022-03-28T14:10:07.804Z] ++ export HADOOP_CONF_DIR=/etc/hadoop
> [2022-03-28T14:10:07.804Z] ++ HADOOP_CONF_DIR=/etc/hadoop
> [2022-03-28T14:10:07.804Z] ++ export HADOOP_LOG_DIR=/data/log
> [2022-03-28T14:10:07.804Z] ++ HADOOP_LOG_DIR=/data/log
> [2022-03-28T14:10:07.804Z] ++ export
> 'HADOOP_CLASSPATH=/etc/tez/:/active/tez/lib/*:/active/tez/*:/apps/lib/*'
> [2022-03-28T14:10:07.804Z] ++
> HADOOP_CLASSPATH='/etc/tez/:/active/tez/lib/*:/active/tez/*:/apps/lib/*'
> [2022-03-28T14:10:07.804Z] ++ export HIVE_CONF_DIR=/etc/hive/
> [2022-03-28T14:10:07.804Z] ++ HIVE_CONF_DIR=/etc/hive/
> [2022-03-28T14:10:07.804Z] ++ export
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/active/hive/bin:/active/hadoop/bin:/active/eclipse/:/active/maven/bin/:/active/protobuf/bin:/active/visualvm/bin:/active/kubebuilder/bin:/active/idea/bin
> [2022-03-28T14:10:07.804Z] ++
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/active/hive/bin:/active/hadoop/bin:/active/eclipse/:/active/maven/bin/:/active/protobuf/bin:/active/visualvm/bin:/active/kubebuilder/bin:/active/idea/bin
> [2022-03-28T14:10:07.804Z] ++ . /etc/profile.d/java.sh
> [2022-03-28T14:10:07.804Z] +++ export JAVA_HOME=/usr/lib/jvm/zulu-8-amd64/
> [2022-03-28T14:10:07.804Z] +++ JAVA_HOME=/usr/lib/jvm/zulu-8-amd64/
> [2022-03-28T14:10:07.804Z] + sw hive-dev
> /home/jenkins/agent/workspace/hive-precommit_PR-3147
> [2022-03-28T14:10:07.804Z] @ activating:
> /home/jenkins/agent/workspace/hive-precommit_PR-3147/packaging/target/apache-hive-4.0.0-alpha-1-SNAPSHOT-bin/apache-hive-4.0.0-alpha-1-SNAPSHOT-bin/
> for hive
> [2022-03-28T14:10:07.804Z] + ping -c2 dev_oracle
> [2022-03-28T14:10:07.804Z] PING dev_oracle (127.0.0.1) 56(84) bytes of data.
> [2022-03-28T14:10:07.804Z] 64 bytes from localhost (127.0.0.1): icmp_seq=1
> ttl=64 time=0.082 ms
> [2022-03-28T14:10:08.795Z] 64 bytes from localhost (127.0.0.1): icmp_seq=2
> ttl=64 time=0.087 ms
> [2022-03-28T14:10:08.795Z]
> [2022-03-28T14:10:08.795Z] --- dev_oracle ping statistics ---
> [2022-03-28T14:10:08.795Z] 2 packets transmitted, 2 received, 0% packet loss,
> time 51ms
> [2022-03-28T14:10:08.795Z] rtt min/avg/max/mdev = 0.082/0.084/0.087/0.009 ms
> [2022-03-28T14:10:08.795Z] + export DOCKER_NETWORK=host
> [2022-03-28T14:10:08.795Z] + DOCKER_NETWORK=host
> [2022-03-28T14:10:08.795Z] + export DBNAME=metastore
> [2022-03-28T14:10:08.795Z] + DBNAME=metastore
> [2022-03-28T14:10:08.795Z] + reinit_metastore oracle
> [2022-03-28T14:10:08.795Z] @ initializing: oracle
> [2022-03-28T14:10:08.795Z] metastore database name: metastore
> [2022-03-28T14:10:09.135Z] @ starting dev_oracle...
> [2022-03-28T14:10:09.445Z] Unable to find image
> 'quay.io/maksymbilenko/oracle-12c:latest' locally
> [2022-03-28T14:10:10.407Z] latest: Pulling from maksymbilenko/oracle-12c
> [2022-03-28T14:10:10.407Z] 8ba884070f61: Pulling fs layer
> [2022-03-28T14:10:10.407Z] ef9513b81046: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 6f1de349e202: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 5376ebfa0fa3: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 5f632c3633d2: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 3e74293031d2: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 5376ebfa0fa3: Waiting
> [2022-03-28T14:10:10.407Z] 5f632c3633d2: Waiting
> [2022-03-28T14:10:10.407Z] 3e74293031d2: Waiting
> [2022-03-28T14:10:10.407Z] 6f1de349e202: Download complete
> [2022-03-28T14:10:11.365Z] ef9513b81046: Download complete
> [2022-03-28T14:10:11.365Z] 5f632c3633d2:
[jira] [Comment Edited] (HIVE-26255) Drop custom code for String deduplication inside URI class
[
https://issues.apache.org/jira/browse/HIVE-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540506#comment-17540506
]
Sylwester Lachiewicz edited comment on HIVE-26255 at 5/21/22 9:28 PM:
--
Also, if we use newer Java - few tests will fail
Unable to make field private volatile java.lang.String java.net.URI.string
accessible: module java.base does not "opens java.net" to unnamed module
@486bb9ce [in thread "main"] at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
{noformat}
[ERROR]
org.apache.hadoop.hive.hbase.TestHBaseSerDe.testHBaseSerDeWithAvroExternalSchema
Time elapsed: 0.001 s <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hive.common.StringInternUtils
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:330)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:322)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:108)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:129)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:104)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:121)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:104)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspector(AvroObjectInspectorGenerator.java:83)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.(AvroObjectInspectorGenerator.java:56)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateAvroStructFromSchema(HBaseSerDeHelper.java:500)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateAvroStructFromClass(HBaseSerDeHelper.java:474)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateColumnStruct(HBaseSerDeHelper.java:446)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateColumnTypes(HBaseSerDeHelper.java:342)
at
org.apache.hadoop.hive.hbase.ColumnMappings.toTypesString(ColumnMappings.java:84)
at
org.apache.hadoop.hive.hbase.HBaseSerDeParameters.(HBaseSerDeParameters.java:94)
at org.apache.hadoop.hive.hbase.HBaseSerDe.initialize(HBaseSerDe.java:118)
at
org.apache.hadoop.hive.hbase.TestHBaseSerDe.testHBaseSerDeWithAvroExternalSchema(TestHBaseSerDe.java:1346)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:577)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at
org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at
java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
[jira] [Commented] (HIVE-26255) Drop custom code for String deduplication inside URI class
[
https://issues.apache.org/jira/browse/HIVE-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540506#comment-17540506
]
Sylwester Lachiewicz commented on HIVE-26255:
-
Also, if we use newer Java - few tests will fail
{noformat}
[ERROR]
org.apache.hadoop.hive.hbase.TestHBaseSerDe.testHBaseSerDeWithAvroExternalSchema
Time elapsed: 0.001 s <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hive.common.StringInternUtils
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:330)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:322)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:108)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:129)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:104)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:121)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:104)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspector(AvroObjectInspectorGenerator.java:83)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.(AvroObjectInspectorGenerator.java:56)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateAvroStructFromSchema(HBaseSerDeHelper.java:500)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateAvroStructFromClass(HBaseSerDeHelper.java:474)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateColumnStruct(HBaseSerDeHelper.java:446)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateColumnTypes(HBaseSerDeHelper.java:342)
at
org.apache.hadoop.hive.hbase.ColumnMappings.toTypesString(ColumnMappings.java:84)
at
org.apache.hadoop.hive.hbase.HBaseSerDeParameters.(HBaseSerDeParameters.java:94)
at org.apache.hadoop.hive.hbase.HBaseSerDe.initialize(HBaseSerDe.java:118)
at
org.apache.hadoop.hive.hbase.TestHBaseSerDe.testHBaseSerDeWithAvroExternalSchema(TestHBaseSerDe.java:1346)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:577)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at
org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at
java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at
[jira] [Resolved] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-24470. - Resolution: Fixed > Separate HiveMetastore Thrift and Driver logic > -- > > Key: HIVE-24470 > URL: https://issues.apache.org/jira/browse/HIVE-24470 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Fix For: storage-2.7.3, storage-2.8.1, 4.0.0-alpha-1 > > Time Spent: 6h 40m > Remaining Estimate: 0h > > In the file HiveMetastore.java the majority of the code is a thrift interface > rather than the actual logic behind starting hive metastore, this should be > moved out into a separate file to clean up the file. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-24470: Fix Version/s: storage-2.7.3 storage-2.8.1 > Separate HiveMetastore Thrift and Driver logic > -- > > Key: HIVE-24470 > URL: https://issues.apache.org/jira/browse/HIVE-24470 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Fix For: storage-2.7.3, storage-2.8.1, 4.0.0-alpha-1 > > Time Spent: 6h 40m > Remaining Estimate: 0h > > In the file HiveMetastore.java the majority of the code is a thrift interface > rather than the actual logic behind starting hive metastore, this should be > moved out into a separate file to clean up the file. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-24470: Fix Version/s: 4.0.0-alpha-1 > Separate HiveMetastore Thrift and Driver logic > -- > > Key: HIVE-24470 > URL: https://issues.apache.org/jira/browse/HIVE-24470 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Time Spent: 6h 40m > Remaining Estimate: 0h > > In the file HiveMetastore.java the majority of the code is a thrift interface > rather than the actual logic behind starting hive metastore, this should be > moved out into a separate file to clean up the file. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HIVE-17632) Build Hive with JDK9
[ https://issues.apache.org/jira/browse/HIVE-17632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-17632. - Resolution: Abandoned > Build Hive with JDK9 > > > Key: HIVE-17632 > URL: https://issues.apache.org/jira/browse/HIVE-17632 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure >Affects Versions: 3.0.0 >Reporter: Haifeng Chen >Priority: Major > Labels: build > > JDK 9 has been released recently with a lot of improvements such as the > support of AVX 512 which can bring performance benefits running on Skylake > servers. > We would expect that the users will soon to try JDK9 and will build Hadoop on > it. Currently it's not clear what issues will the user have to build Hive on > JDK9. The JIRA can serve as the umbrella JIRA to track all these issues. > http://jdk.java.net/9/ -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26092) Fix javadoc errors for the 4.0.0 release
[ https://issues.apache.org/jira/browse/HIVE-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538173#comment-17538173 ] Sylwester Lachiewicz commented on HIVE-26092: - PR 3185 merged to 4.0 [https://github.com/apache/hive/commit/53784e08000ebc3364be5c0fb2d796305b7b6844] > Fix javadoc errors for the 4.0.0 release > > > Key: HIVE-26092 > URL: https://issues.apache.org/jira/browse/HIVE-26092 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-2 > > Time Spent: 50m > Remaining Estimate: 0h > > Currently there are plenty of errors in the javadoc. > We should fix those before a final release -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-26092) Fix javadoc errors for the 4.0.0 release
[ https://issues.apache.org/jira/browse/HIVE-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26092: Fix Version/s: 4.0.0-alpha-2 > Fix javadoc errors for the 4.0.0 release > > > Key: HIVE-26092 > URL: https://issues.apache.org/jira/browse/HIVE-26092 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-2 > > Time Spent: 50m > Remaining Estimate: 0h > > Currently there are plenty of errors in the javadoc. > We should fix those before a final release -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-26222) Native GeoSpatial Support in Hive
[ https://issues.apache.org/jira/browse/HIVE-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26222: Description: This is an epic Jira to support GeoSpatial datatypes natively in Hive. This will cater to the applications which queries on large volumes of spatial data. This support will be added in a phased manner. To start with, we are planning to make use of framework developed by ESRI [https://github.com/Esri/spatial-framework-for-hadoop |https://github.com/Esri/spatial-framework-for-hadoop).] This project is not very active and there is no release done to maven central. So its not easy to get the jars downloaded directly using pom dependency. Also the UDFs are based on older version of Hive. So we have decided to make a copy of this repo and maintained it inside Hive. This will make it easier to do any improvement and manage dependencies. As of now, the data loading is done only on a binary data type. We need to enhance this to make it more user friendly. In the next phase, a native Geometry/Geography datatype will be supported. User can directly create a geometry type and operate on it. Apart from these we can start adding support for different indices like quad tree, R-tree, ORC/Parquet/Iceberg support etc. (was: This is an epic Jira to support GeoSpatial datatypes natively in Hive. This will cater to the applications which queries on large volumes of spatial data. This support will be added in a phased manner. To start with, we are planning to make use of framework developed by ESRI ([https://github.com/Esri/spatial-framework-for-hadoop).] This project is not very active and there is no release done to maven central. So its not easy to get the jars downloaded directly using pom dependency. Also the UDFs are based on older version of Hive. So we have decided to make a copy of this repo and maintained it inside Hive. This will make it easier to do any improvement and manage dependencies. As of now, the data loading is done only on a binary data type. We need to enhance this to make it more user friendly. In the next phase, a native Geometry/Geography datatype will be supported. User can directly create a geometry type and operate on it. Apart from these we can start adding support for different indices like quad tree, R-tree, ORC/Parquet/Iceberg support etc. ) > Native GeoSpatial Support in Hive > - > > Key: HIVE-26222 > URL: https://issues.apache.org/jira/browse/HIVE-26222 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > > This is an epic Jira to support GeoSpatial datatypes natively in Hive. This > will cater to the applications which queries on large volumes of spatial > data. This support will be added in a phased manner. To start with, we are > planning to make use of framework developed by ESRI > [https://github.com/Esri/spatial-framework-for-hadoop > |https://github.com/Esri/spatial-framework-for-hadoop).] This project is not > very active and there is no release done to maven central. So its not easy to > get the jars downloaded directly using pom dependency. Also the UDFs are > based on older version of Hive. So we have decided to make a copy of this > repo and maintained it inside Hive. This will make it easier to do any > improvement and manage dependencies. As of now, the data loading is done only > on a binary data type. We need to enhance this to make it more user > friendly. In the next phase, a native Geometry/Geography datatype will be > supported. User can directly create a geometry type and operate on it. Apart > from these we can start adding support for different indices like quad tree, > R-tree, ORC/Parquet/Iceberg support etc. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-26222) Native GeoSpatial Support in Hive
[ https://issues.apache.org/jira/browse/HIVE-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26222: Description: This is an epic Jira to support GeoSpatial datatypes natively in Hive. This will cater to the applications which queries on large volumes of spatial data. This support will be added in a phased manner. To start with, we are planning to make use of framework developed by ESRI [https://github.com/Esri/spatial-framework-for-hadoop |https://github.com/Esri/spatial-framework-for-hadoop).] on the Apache 2.0 license. This project is not very active and there is no release done to maven central. So its not easy to get the jars downloaded directly using pom dependency. Also the UDFs are based on older version of Hive. So we have decided to make a copy of this repo and maintained it inside Hive. This will make it easier to do any improvement and manage dependencies. As of now, the data loading is done only on a binary data type. We need to enhance this to make it more user friendly. In the next phase, a native Geometry/Geography datatype will be supported. User can directly create a geometry type and operate on it. Apart from these we can start adding support for different indices like quad tree, R-tree, ORC/Parquet/Iceberg support etc. (was: This is an epic Jira to support GeoSpatial datatypes natively in Hive. This will cater to the applications which queries on large volumes of spatial data. This support will be added in a phased manner. To start with, we are planning to make use of framework developed by ESRI [https://github.com/Esri/spatial-framework-for-hadoop |https://github.com/Esri/spatial-framework-for-hadoop).] This project is not very active and there is no release done to maven central. So its not easy to get the jars downloaded directly using pom dependency. Also the UDFs are based on older version of Hive. So we have decided to make a copy of this repo and maintained it inside Hive. This will make it easier to do any improvement and manage dependencies. As of now, the data loading is done only on a binary data type. We need to enhance this to make it more user friendly. In the next phase, a native Geometry/Geography datatype will be supported. User can directly create a geometry type and operate on it. Apart from these we can start adding support for different indices like quad tree, R-tree, ORC/Parquet/Iceberg support etc. ) > Native GeoSpatial Support in Hive > - > > Key: HIVE-26222 > URL: https://issues.apache.org/jira/browse/HIVE-26222 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > > This is an epic Jira to support GeoSpatial datatypes natively in Hive. This > will cater to the applications which queries on large volumes of spatial > data. This support will be added in a phased manner. To start with, we are > planning to make use of framework developed by ESRI > [https://github.com/Esri/spatial-framework-for-hadoop > |https://github.com/Esri/spatial-framework-for-hadoop).] on the Apache 2.0 > license. This project is not very active and there is no release done to > maven central. So its not easy to get the jars downloaded directly using pom > dependency. Also the UDFs are based on older version of Hive. So we have > decided to make a copy of this repo and maintained it inside Hive. This will > make it easier to do any improvement and manage dependencies. As of now, the > data loading is done only on a binary data type. We need to enhance this to > make it more user friendly. In the next phase, a native Geometry/Geography > datatype will be supported. User can directly create a geometry type and > operate on it. Apart from these we can start adding support for different > indices like quad tree, R-tree, ORC/Parquet/Iceberg support etc. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (HIVE-26226) Exclude jdk.tools dep from hive-metastore in upgrade-acid
[ https://issues.apache.org/jira/browse/HIVE-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz reassigned HIVE-26226: --- Assignee: Sylwester Lachiewicz > Exclude jdk.tools dep from hive-metastore in upgrade-acid > - > > Key: HIVE-26226 > URL: https://issues.apache.org/jira/browse/HIVE-26226 > Project: Hive > Issue Type: Improvement > Components: Tests >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sylwester Lachiewicz >Assignee: Sylwester Lachiewicz >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-2 > > Attachments: jdktools_deps_master.txt > > Time Spent: 10m > Remaining Estimate: 0h > > The hive-metastore 2.3.3 used in upgrade-acid tests includes unnecessary > dependency - that blocks the possibility to compile with newer java versions > > 8 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26226) Exclude jdk.tools dep from hive-metastore in upgrade-acid
[ https://issues.apache.org/jira/browse/HIVE-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538149#comment-17538149 ] Sylwester Lachiewicz commented on HIVE-26226: - Thx. sure, I'll check and rebase PR. > Exclude jdk.tools dep from hive-metastore in upgrade-acid > - > > Key: HIVE-26226 > URL: https://issues.apache.org/jira/browse/HIVE-26226 > Project: Hive > Issue Type: Improvement > Components: Tests >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sylwester Lachiewicz >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-2 > > Attachments: jdktools_deps_master.txt > > Time Spent: 10m > Remaining Estimate: 0h > > The hive-metastore 2.3.3 used in upgrade-acid tests includes unnecessary > dependency - that blocks the possibility to compile with newer java versions > > 8 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26226) Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid)
[ https://issues.apache.org/jira/browse/HIVE-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17536745#comment-17536745 ] Sylwester Lachiewicz commented on HIVE-26226: - thx for output - now I can also reproduce output with Maven 3.8.1 and Java 8. So, only hbase-annotations 1.1.1 have a direct dependency on jdk.tools 1.7. All others have dependency defined in the profile for Java 7 or 8. Because we compile with Java 11 dependency isn't required. It's safe to exclude one from hive-metastore (2.3.3) because here, inside upgrade-acid we also have 2nd instance that comes from hadoop-common (2.7.2) - hadoop-annotations and then jdk.tools with 2 profiles (7,8) > Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid) > > > Key: HIVE-26226 > URL: https://issues.apache.org/jira/browse/HIVE-26226 > Project: Hive > Issue Type: Improvement > Components: Tests >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sylwester Lachiewicz >Priority: Minor > Attachments: jdktools_deps_master.txt > > > The hive-metastore 2.3.3 used in upgrade-acid tests includes unnecessary > dependency - that blocks the possibility to compile with newer java versions > > 8 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26226) Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid)
[ https://issues.apache.org/jira/browse/HIVE-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17536649#comment-17536649 ] Sylwester Lachiewicz commented on HIVE-26226: - I'm blind or have an issue with my env - but I found only one and fixed this PR #3284 would you be able to add more commits where this dependency exists? Also - an alternative way to fix this may be to update the used HBASE from 1.1.1 to 1.2.0 but then the impact may be bigger. > Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid) > > > Key: HIVE-26226 > URL: https://issues.apache.org/jira/browse/HIVE-26226 > Project: Hive > Issue Type: Improvement > Components: Tests >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sylwester Lachiewicz >Priority: Minor > > The hive-metastore 2.3.3 used in upgrade-acid tests includes unnecessary > dependency - that blocks the possibility to compile with newer java versions > > 8 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26226) Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid)
[
https://issues.apache.org/jira/browse/HIVE-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17536140#comment-17536140
]
Sylwester Lachiewicz commented on HIVE-26226:
-
{code}
hive-metastore 2.3.3 (but also 2.3.9) hbase-client 1.1.1 -> hbase-annotations
1.1.1 (up to 1.21.13)-> jdk.tools 1.7 (system)
{code}
jdk.tools was dropped with hbase-annotations 1.2.0
> Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid)
>
>
> Key: HIVE-26226
> URL: https://issues.apache.org/jira/browse/HIVE-26226
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 3.1.3, 4.0.0-alpha-2
>Reporter: Sylwester Lachiewicz
>Priority: Minor
>
> The hive-metastore 2.3.3 used in upgrade-acid tests includes unnecessary
> dependency - that blocks the possibility to compile with newer java versions
> > 8
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HIVE-26226) Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid)
[ https://issues.apache.org/jira/browse/HIVE-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17536121#comment-17536121 ] Sylwester Lachiewicz commented on HIVE-26226: - I found this issue only inside upgrade-acid - that prevents me from loading projects with my IDE. I'm using Maven 3.8.5 and Java 18 so even more challenges. I don't this small update would affect executing tests (CI is green) - all code still will stay compiled for Java 8 as before. Where have You found other jdk.tools mvn org.apache.maven.plugins:maven-dependency-plugin:3.3.0:tree -Dincludes=jdk.tools:jdk.tools doesn't show anything more. I'm checking branch-3.1 now. btw. [https://github.com/apache/hive/pull/3286] > Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid) > > > Key: HIVE-26226 > URL: https://issues.apache.org/jira/browse/HIVE-26226 > Project: Hive > Issue Type: Improvement > Components: Tests >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sylwester Lachiewicz >Priority: Minor > > The hive-metastore 2.3.3 used in upgrade-acid tests includes unnecessary > dependency - that blocks the possibility to compile with newer java versions > > 8 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26226) Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid)
[ https://issues.apache.org/jira/browse/HIVE-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17536100#comment-17536100 ] Sylwester Lachiewicz commented on HIVE-26226: - yes, I was convinced that I had seen this problem already in someone, but I did not find it in Jira. I simply excluded this dependency (see linked PR for 3.1.x and 4.x) > Remove outdated dependency to jdk.tools:jdk.tools:jar:1.7 (upgrade-acid) > > > Key: HIVE-26226 > URL: https://issues.apache.org/jira/browse/HIVE-26226 > Project: Hive > Issue Type: Improvement > Components: Tests >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sylwester Lachiewicz >Priority: Minor > > The hive-metastore 2.3.3 used in upgrade-acid tests includes unnecessary > dependency - that blocks the possibility to compile with newer java versions > > 8 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] (HIVE-26218) Hive standalone metastore release version are not available after 3.0.0 in offical download location
[ https://issues.apache.org/jira/browse/HIVE-26218 ] Sylwester Lachiewicz deleted comment on HIVE-26218: - was (Author: slachiewicz): Can we create separate repo and move there hive standalone, as mentioned on mailing list? This could simplify releases > Hive standalone metastore release version are not available after 3.0.0 in > offical download location > > > Key: HIVE-26218 > URL: https://issues.apache.org/jira/browse/HIVE-26218 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.2, 3.1.3, 4.0.0, 4.0.0-alpha-1 >Reporter: Nagarajan Selvaraj >Priority: Major > > Hive standalone metastore release version are not available after 3.0.0 in > official download location https://www.apache.org/dyn/closer.cgi/hive/ -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26218) Hive standalone metastore release version are not available after 3.0.0 in offical download location
[ https://issues.apache.org/jira/browse/HIVE-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17535019#comment-17535019 ] Sylwester Lachiewicz commented on HIVE-26218: - Can we create separate repo and move there hive standalone, as mentioned on mailing list? This could simplify releases > Hive standalone metastore release version are not available after 3.0.0 in > offical download location > > > Key: HIVE-26218 > URL: https://issues.apache.org/jira/browse/HIVE-26218 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.2, 3.1.3, 4.0.0, 4.0.0-alpha-1 >Reporter: Nagarajan Selvaraj >Priority: Major > > Hive standalone metastore release version are not available after 3.0.0 in > official download location https://www.apache.org/dyn/closer.cgi/hive/ -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HIVE-26063) Upgrade Apache parent POM to version 25
[ https://issues.apache.org/jira/browse/HIVE-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26063: Fix Version/s: (was: 4.0.0) (was: 4.0.0-alpha-1) > Upgrade Apache parent POM to version 25 > --- > > Key: HIVE-26063 > URL: https://issues.apache.org/jira/browse/HIVE-26063 > Project: Hive > Issue Type: Improvement >Reporter: Sylwester Lachiewicz >Priority: Major > > [https://maven.apache.org/pom/] 25 has been released on 2022-02-20 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26063) Upgrade Apache parent POM to version 25
[ https://issues.apache.org/jira/browse/HIVE-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26063: Description: [https://maven.apache.org/pom/] 25 has been released on 2022-02-20 > Upgrade Apache parent POM to version 25 > --- > > Key: HIVE-26063 > URL: https://issues.apache.org/jira/browse/HIVE-26063 > Project: Hive > Issue Type: Improvement >Reporter: Sylwester Lachiewicz >Priority: Major > Fix For: 4.0.0, 4.0.0-alpha-1 > > > [https://maven.apache.org/pom/] 25 has been released on 2022-02-20 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26063) Upgrade Apache parent POM to version 25
[ https://issues.apache.org/jira/browse/HIVE-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-26063: Summary: Upgrade Apache parent POM to version 25 (was: Upgrade Apache parent POM to version 35) > Upgrade Apache parent POM to version 25 > --- > > Key: HIVE-26063 > URL: https://issues.apache.org/jira/browse/HIVE-26063 > Project: Hive > Issue Type: Improvement >Reporter: Sylwester Lachiewicz >Priority: Major > Fix For: 4.0.0, 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-26063) Upgrade Apache parent POM to version 35
[ https://issues.apache.org/jira/browse/HIVE-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz reassigned HIVE-26063: --- Assignee: (was: David Mollitor) > Upgrade Apache parent POM to version 35 > --- > > Key: HIVE-26063 > URL: https://issues.apache.org/jira/browse/HIVE-26063 > Project: Hive > Issue Type: Improvement >Reporter: Sylwester Lachiewicz >Priority: Major > Fix For: 4.0.0, 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-26063) Upgrade Apache parent POM to version 35
[ https://issues.apache.org/jira/browse/HIVE-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz reassigned HIVE-26063: --- > Upgrade Apache parent POM to version 35 > --- > > Key: HIVE-26063 > URL: https://issues.apache.org/jira/browse/HIVE-26063 > Project: Hive > Issue Type: Improvement >Reporter: Sylwester Lachiewicz >Assignee: David Mollitor >Priority: Major > Fix For: 4.0.0, 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495260#comment-17495260 ] Sylwester Lachiewicz commented on HIVE-24470: - merged with [b0309b7f023d9785c3a842d70d0fc471252101bf|https://github.com/apache/hive/commit/b0309b7f023d9785c3a842d70d0fc471252101bf] > Separate HiveMetastore Thrift and Driver logic > -- > > Key: HIVE-24470 > URL: https://issues.apache.org/jira/browse/HIVE-24470 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Time Spent: 6h 40m > Remaining Estimate: 0h > > In the file HiveMetastore.java the majority of the code is a thrift interface > rather than the actual logic behind starting hive metastore, this should be > moved out into a separate file to clean up the file. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-24483) Bump protobuf version to 3.12.0 (aarch64 support)
[ https://issues.apache.org/jira/browse/HIVE-24483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-24483: Summary: Bump protobuf version to 3.12.0 (aarch64 support) (was: Bump protobuf version to 3.12.0) > Bump protobuf version to 3.12.0 (aarch64 support) > - > > Key: HIVE-24483 > URL: https://issues.apache.org/jira/browse/HIVE-24483 > Project: Hive > Issue Type: Improvement >Reporter: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-25536) Upgrade to Kafka 2.8
[ https://issues.apache.org/jira/browse/HIVE-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz updated HIVE-25536: Labels: (was: pull-request-available) > Upgrade to Kafka 2.8 > > > Key: HIVE-25536 > URL: https://issues.apache.org/jira/browse/HIVE-25536 > Project: Hive > Issue Type: Improvement > Components: kafka integration >Reporter: Viktor Somogyi-Vass >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HIVE-25855) Make a branch-3 release
[ https://issues.apache.org/jira/browse/HIVE-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495252#comment-17495252 ] Sylwester Lachiewicz commented on HIVE-25855: - can I help somehow? > Make a branch-3 release > > > Key: HIVE-25855 > URL: https://issues.apache.org/jira/browse/HIVE-25855 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > > This jira is to track commits for a hive release off branch-3 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HIVE-25876) Update log4j2 version to 2.17.1
[ https://issues.apache.org/jira/browse/HIVE-25876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylwester Lachiewicz resolved HIVE-25876. - Resolution: Duplicate > Update log4j2 version to 2.17.1 > --- > > Key: HIVE-25876 > URL: https://issues.apache.org/jira/browse/HIVE-25876 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 3.1.2 >Reporter: Anatoly >Priority: Blocker > > Hive version 3.1.2 -> log2j -> Should upgrade the version to 2.17.1 -- This message was sent by Atlassian Jira (v8.20.1#820001)
