Re: Release timing for 3.1.2?

[Kevin Marr]

Hi Alan, that would be greatly appreciated! I'm not a developer so I won't
be able to fix anything myself, but I'm happy to test the release and
report any bugs I find.

On Fri, Aug 9, 2019 at 2:25 PM Alan Gates  wrote:

> I'm not aware of any discussions to push a 3.1.2 release.  I'm can work on
> putting together a release of what's currently in the 3.1.2 line.  If we
> hit issues with tests not passing or other such things are you available to
> help?
>
> Alan.
>
> On Mon, Jul 29, 2019 at 6:18 PM Kevin Marr 
> wrote:
>
> > Hello Hive Dev Community,
> >
> > I'm Kevin, a Product Manager at Looker (recently acquired by Google). We
> > make business intelligence software and connect to 40+ relational data
> > warehouses, including Hive.
> >
> > Recently we've been struggling with a Hive bug (HIVE-18624
> > ), where parsing time
> is
> > extremely high for complex SELECT expressions. This is problematic for us
> > because Looker can sometimes generate very complex SQL to represent our
> > customer's reporting logic (we do querying in-database rather than
> > ingesting data and manipulating it in proprietary systems).
> >
> > It appears that the bug has been fixed for version 3.1.2 but has not yet
> > been released. Would it be possible to make a release of 3.1.2 so that we
> > and our customers can take advantage of the bug fix?
> >
> > Thank you,
> > Kevin
> >
>

[jira] [Created] (HIVE-22095) Hive.get() resets the capabilities from HiveConf instead of set capabilities

[Naveen Gangam (JIRA)]

Naveen Gangam created HIVE-22095:


 Summary: Hive.get() resets the capabilities from HiveConf instead 
of set capabilities
 Key: HIVE-22095
 URL: https://issues.apache.org/jira/browse/HIVE-22095
 Project: Hive
  Issue Type: Sub-task
  Components: Hive
Affects Versions: 4.0.0
Reporter: Naveen Gangam
Assignee: Naveen Gangam


Hive.get() resets the capabilities set on the HiveMetaStoreClient from what is 
set in HiveConf instead of preserving the capabilities that have already been 
set via setHMSClientCapabilties()



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: Question about feasibility of porting HIVE-21584 and minor release 2.3.6

[Alan Gates]

I think it's fine to backport this and do a release.  Are we sure that's
enough to make it run on JDK11?  As noted in the bug there isn't an
umbrella issue to make it JDK11 compatible.  I don't know if anyone has
tested Hive 2 on JDK11 or not.

Are you available to do the backport?  If so, and we don't find any other
JDK11 related issues, I can create a 2.3.6 release once you're done.

Alan.

On Thu, Aug 8, 2019 at 6:23 PM Hyukjin Kwon  wrote:

> Hi all,
>
>
> I am from Spark dev and had a question about feasibility of porting
> HIVE-21584
> and minor release 2.3.x.
>
> Just to share full context, please take a look at
>
> https://issues.apache.org/jira/browse/HIVE-21639?focusedCommentId=16822802=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16822802
> TL;DR: with Hive 2.3.5, Spark is now very close to get through the
> complicated
> situation that has been locked for years. Thank you Yuming Wang and Alan
> Gates
> - it would have been impossible without all the efforts from you guys.
>
>
> One problem found lately though,
> Spark is trying to support JDK 11 but this seems blocked by HIVE-21584.
> So, I and Spark
> community are trying to find a way to get through.
>
> Firstly, I (and presumably some Spark community guys) thought Spark should
> find its
> own workaround or try to upgrade Hive to 4.0.0 in the upcoming release.
> This possibility was checked and seems difficult.
> It was already pretty a radical change to upgrade Hive 1.2.x to 2.3.5,
> seems it's difficult to target upgrade to 4.0.0 in this release at this
> moment.
>
> I understand HIVE-21584 was fixed in 4.0.0 and it might be difficult to
> backport
> through branch-3.x and 2.x; however, wanted to at least ask and now because
> I thought porting HIVE-21584 into Hive 2.3.x branch is an option that
> needs the
> minimised efforts to permanently resolve all related issues, apparently,
> blocked
> for multiple years ..
>
>
> Thanks for consideration in advance.
>

Re: Release timing for 3.1.2?

[Alan Gates]

I'm not aware of any discussions to push a 3.1.2 release.  I'm can work on
putting together a release of what's currently in the 3.1.2 line.  If we
hit issues with tests not passing or other such things are you available to
help?

Alan.

On Mon, Jul 29, 2019 at 6:18 PM Kevin Marr  wrote:

> Hello Hive Dev Community,
>
> I'm Kevin, a Product Manager at Looker (recently acquired by Google). We
> make business intelligence software and connect to 40+ relational data
> warehouses, including Hive.
>
> Recently we've been struggling with a Hive bug (HIVE-18624
> ), where parsing time is
> extremely high for complex SELECT expressions. This is problematic for us
> because Looker can sometimes generate very complex SQL to represent our
> customer's reporting logic (we do querying in-database rather than
> ingesting data and manipulating it in proprietary systems).
>
> It appears that the bug has been fixed for version 3.1.2 but has not yet
> been released. Would it be possible to make a release of 3.1.2 so that we
> and our customers can take advantage of the bug fix?
>
> Thank you,
> Kevin
>

Review Request 71262: Mondrian queries failing with ClassCastException: hive.ql.exec.vector.DecimalColumnVector cannot be cast to hive.ql.exec.vector.Decimal64ColumnVector

[Attila Magyar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71262/
---

Review request for hive, Ashutosh Chauhan, Gopal V, and Jesús Camacho Rodríguez.


Bugs: HIVE-22094
https://issues.apache.org/jira/browse/HIVE-22094


Repository: hive-git


Description
---

ClassNotFoundException when running join on decimal column:

Caused by: java.lang.ClassCastException: 
org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector cannot be cast to 
org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector

at 
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFSumDecimal64ToDecimal.aggregateInput(VectorUDAFSumDecimal64ToDecimal.java:320)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:217)


Diffs
-

  data/files/employee_closure/employee_closure.tsv PRE-CREATION 
  data/files/salary/salary.tsv PRE-CREATION 
  itests/src/test/resources/testconfiguration.properties 84c20426763 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinCommonOperator.java
 573368829e5 
  ql/src/test/queries/clientpositive/vector_decimal_mapjoin2.q PRE-CREATION 
  ql/src/test/results/clientpositive/llap/vector_decimal_mapjoin2.q.out 
PRE-CREATION 


Diff: https://reviews.apache.org/r/71262/diff/1/


Testing
---

qtest


Thanks,

Attila Magyar

[jira] [Created] (HIVE-22094) Mondrian queries failing with ClassCastException: hive.ql.exec.vector.DecimalColumnVector cannot be cast to hive.ql.exec.vector.Decimal64ColumnVector

[Attila Magyar (JIRA)]

Attila Magyar created HIVE-22094:


 Summary: Mondrian queries failing with ClassCastException: 
hive.ql.exec.vector.DecimalColumnVector cannot be cast to 
hive.ql.exec.vector.Decimal64ColumnVector
 Key: HIVE-22094
 URL: https://issues.apache.org/jira/browse/HIVE-22094
 Project: Hive
  Issue Type: Task
  Components: Hive
Reporter: Attila Magyar
Assignee: Attila Magyar
 Fix For: 4.0.0


When running a query like this

select sum(salary.salary_paid) from salary, employee_closure where 
salary.employee_id = employee_closure.employee_id;

with hive.auto.convert.join=true and hive.vectorized.execution.enabled=true the 
following exception occurs
{code:java}
Caused by: java.lang.ClassCastException: 
org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector cannot be cast to 
org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector

at 
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFSumDecimal64ToDecimal.aggregateInput(VectorUDAFSumDecimal64ToDecimal.java:320)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:217)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.doProcessBatch(VectorGroupByOperator.java:414)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processBatch(VectorGroupByOperator.java:182)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.process(VectorGroupByOperator.java:1124)

at org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:919)

at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.forwardOverflow(VectorMapJoinGenerateResultOperator.java:706)

at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerBigOnlyGenerateResultOperator.generateHashMultiSetResultMultiValue(VectorMapJoinInnerBigOnlyGenerateResultOperator.java:268)

at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerBigOnlyGenerateResultOperator.finishInnerBigOnly(VectorMapJoinInnerBigOnlyGenerateResultOperator.java:180)

at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerBigOnlyLongOperator.processBatch(VectorMapJoinInnerBigOnlyLongOperator.java:379)

... 28 more{code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Created] (HIVE-22093) JdbcStorageHandler query mysql with wrong offset

[wang dawei (JIRA)]

wang dawei created HIVE-22093:
-

 Summary: JdbcStorageHandler query mysql with wrong offset
 Key: HIVE-22093
 URL: https://issues.apache.org/jira/browse/HIVE-22093
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.3.5
Reporter: wang dawei


CREATE EXTERNAL TABLE student_jdbc
(
 name string,
 age int,
 gpa double
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
 "hive.sql.database.type" = "MYSQL",
 "hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
 "hive.sql.jdbc.url" = "jdbc:mysql://localhost:3306/test",
 "hive.sql.dbcp.username" = "xxx",
 "hive.sql.dbcp.password" = "x",
 "hive.sql.table" = "student_1",
 "hive.sql.partitionColumn" = "id",
 "hive.sql.numPartitions" = "5",
 "hive.sql.lowerBound" = "1",
 "hive.sql.upperBound" = "36",
 "hive.sql.dbcp.maxActive" = "5",
 "hive.sql.query" = "SELECT name, age, gpa FROM student_1"
);



set mapred.map.tasks=5;

select count(*) from student_jdbc;

82

mysql:
select count(*) from student;
36



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)