Re: Incorrect Release Notes for Hive-2.3.3

2018-10-17 Thread Daniel Dai
Yes, I cleared fixed version from the Jiras. It should be fixed.

Thanks,
Daniel

From: Oleksiy S 
Reply-To: "u...@hive.apache.org" 
Date: Friday, October 5, 2018 at 5:02 AM
To: "dev@hive.apache.org" , "u...@hive.apache.org" 

Subject: Re: Incorrect Release Notes for Hive-2.3.3

Guys any updates?

On Thu, Oct 4, 2018 at 11:15 AM Oleksiy S 
mailto:osayankin.superu...@gmail.com>> wrote:
Current release notes for Hive-2.3.3 are: 
RNs

Release Notes - Hive - Version 2.3.3

** Bug
* [HIVE-16939] - metastore error: 'export: -Dproc_metastore : not a valid 
identifier'
* [HIVE-18767] - Some alterPartitions invocations throw 
'NumberFormatException: null'
* [HIVE-18788] - Clean up inputs in JDBC PreparedStatement
* [HIVE-18815] - Remove unused feature in HPL/SQL
* [HIVE-18879] - Disallow embedded element in UDFXPathUtil needs to work if 
xercesImpl.jar in classpath
* [HIVE-18885] - DbNotificationListener has a deadlock between Java and DB 
locks (2.x line)
* [HIVE-20304] - When hive.optimize.skewjoin and hive.auto.convert.join are 
both set to true, and the execution engine is mr, same stage may launch twice 
due to the wrong generated plan
* [HIVE-20441] - NPE in ExprNodeGenericFuncDesc  when 
hive.allow.udf.load.on.demand is set to true

** Improvement
* [HIVE-19900] - HiveCLI HoS Performs Invalid Impersonation If User Name 
Truncated
* [HIVE-20284] - In strict mode, if constant propagation is enable, the 
partition filter may be folded before partition pruner lead to error "No 
partition predicate for Alias"


See table for short summary.
Issue

Comment

HIVE-18767

Issue is not in Hive-2.3.3

HIVE-20304

Issue is unresolved in Apache Hive JIRA

HIVE-20441

Issue is unresolved in Apache Hive JIRA

HIVE-19900

Resolved: Workaround. No commit to see

HIVE-20284

Issue is unresolved in Apache Hive JIRA


Please remove incorrect JIRAs from RNs.

--
Oleksiy


--
Oleksiy


[jira] [Created] (HIVE-20770) Need improvement in hive for ACID properties and tables

2018-10-17 Thread pritam (JIRA)
pritam created HIVE-20770:
-

 Summary: Need improvement in hive for ACID properties and tables
 Key: HIVE-20770
 URL: https://issues.apache.org/jira/browse/HIVE-20770
 Project: Hive
  Issue Type: Improvement
  Components: Hive
Reporter: pritam


Hello Team,

In current version of Apache Hive if we set ACID properties it can not be 
revert as well as

Apache Spark does not support Hive ACID table. If it is possible to revert ACID 
properties from hive table and read-write Hive table from spark-scala please 
let me know.

If there is no provision for above conditions it is important to add features 
and improvement in next Apache Hive version. It will be very helpful for all in 
distributed batch processing.

I am eager to hear from you. Thank you all in advance for such a great batch 
processing tool. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] hive pull request #449: HIVE-20679: DDL operations on hive might create larg...

2018-10-17 Thread anishek
GitHub user anishek opened a pull request:

https://github.com/apache/hive/pull/449

HIVE-20679: DDL operations on hive might create large messages for 
DBNotification



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/anishek/hive HIVE-20679

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/449.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #449


commit 9767cec4e7eb9d05afa6ac6f9279a6358cd50815
Author: Anishek Agarwal 
Date:   2018-10-03T09:28:40Z

HIVE-20679: DDL operations on hive might create large messages for 
DBNotification




---


[GitHub] hive pull request #446: HIVE-20679: DDL operations on hive might create larg...

2018-10-17 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/446


---


Re: Review Request 69050: HIVE-20720: Add partition column option to JDBC handler

2018-10-17 Thread Jesús Camacho Rodríguez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69050/#review209734
---


Ship it!




Ship It!

- Jesús Camacho Rodríguez


On Oct. 16, 2018, 5:08 p.m., Daniel Dai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69050/
> ---
> 
> (Updated Oct. 16, 2018, 5:08 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> See HIVE-20720.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/Constants.java 1190679 
>   itests/src/test/resources/testconfiguration.properties 9a87464 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputFormat.java 
> 74999db 
>   jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputSplit.java 
> 3a6ada8 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java 
> 1da6213 
>   jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java 
> 5947628 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java
>  18e2397 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessor.java
>  fdaa794 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java
>  abdc5f0 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java
>  a95aca2 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JethroDatabaseAccessor.java
>  db0454e 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/MySqlDatabaseAccessor.java
>  86fde7c 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DateIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DecimalIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DoubleIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/IntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/IntervalSplitterFactory.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/LongIntervalSpitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/TimestampIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/test/java/org/apache/hive/storage/jdbc/TestJdbcInputFormat.java
>  b146633 
>   
> jdbc-handler/src/test/java/org/apache/hive/storage/jdbc/dao/TestGenericJdbcDatabaseAccessor.java
>  34f061e 
>   ql/src/test/queries/clientpositive/external_jdbc_table_partition.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/external_jdbc_table_typeconversion.q 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/external_jdbc_table_partition.q.out 
> PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/external_jdbc_table_typeconversion.q.out
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69050/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Daniel Dai
> 
>



Re: Review Request 69050: HIVE-20720: Add partition column option to JDBC handler

2018-10-17 Thread Daniel Dai


> On Oct. 17, 2018, 8:29 p.m., Jesús Camacho Rodríguez wrote:
> > jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java
> > Lines 233 (patched)
> > 
> >
> > Could we use a case insensitive regex that would match something like 
> > 'FROM\s+tableString'? I believe it would be safer. If you do not tackle it 
> > in this issue, let's create a follow-up.

Yes, it is better as it won't match anything before the first FROM. However, it 
could be quite complex to get it absolutely right. We also need to make sure it 
will search after "JOIN" and won't search beyond "WHERE/GROUP" etc. And can 
there be multiple "FROM"? I adopt the "from" part in the patch, but still leave 
the TODO tag and will think about a better way in the future.


- Daniel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69050/#review209714
---


On Oct. 16, 2018, 5:08 p.m., Daniel Dai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69050/
> ---
> 
> (Updated Oct. 16, 2018, 5:08 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> See HIVE-20720.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/Constants.java 1190679 
>   itests/src/test/resources/testconfiguration.properties 9a87464 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputFormat.java 
> 74999db 
>   jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputSplit.java 
> 3a6ada8 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java 
> 1da6213 
>   jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java 
> 5947628 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java
>  18e2397 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessor.java
>  fdaa794 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java
>  abdc5f0 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java
>  a95aca2 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JethroDatabaseAccessor.java
>  db0454e 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/MySqlDatabaseAccessor.java
>  86fde7c 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DateIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DecimalIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DoubleIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/IntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/IntervalSplitterFactory.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/LongIntervalSpitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/TimestampIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/test/java/org/apache/hive/storage/jdbc/TestJdbcInputFormat.java
>  b146633 
>   
> jdbc-handler/src/test/java/org/apache/hive/storage/jdbc/dao/TestGenericJdbcDatabaseAccessor.java
>  34f061e 
>   ql/src/test/queries/clientpositive/external_jdbc_table_partition.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/external_jdbc_table_typeconversion.q 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/external_jdbc_table_partition.q.out 
> PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/external_jdbc_table_typeconversion.q.out
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69050/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Daniel Dai
> 
>



[jira] [Created] (HIVE-20769) TxnHandler.checkLock() will re-acquire the same lock

2018-10-17 Thread Eugene Koifman (JIRA)
Eugene Koifman created HIVE-20769:
-

 Summary: TxnHandler.checkLock() will re-acquire the same lock
 Key: HIVE-20769
 URL: https://issues.apache.org/jira/browse/HIVE-20769
 Project: Hive
  Issue Type: Sub-task
  Components: Transactions
Affects Versions: 3.1.0
Reporter: Eugene Koifman


as currently implemented, this will acquire the same type of lock on the same 
resource if requested by another stmt in the same txn.  Need to fix that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20768) Adding Tumbling Window UDF

2018-10-17 Thread slim bouguerra (JIRA)
slim bouguerra created HIVE-20768:
-

 Summary: Adding Tumbling Window UDF
 Key: HIVE-20768
 URL: https://issues.apache.org/jira/browse/HIVE-20768
 Project: Hive
  Issue Type: New Feature
Reporter: slim bouguerra
Assignee: slim bouguerra


Goal is to provide a UDF that truncates a timestamp to a beginning of a 
tumbling window interval.
{code}
/**
 * Tumbling windows are a series of fixed-sized, non-overlapping and contiguous 
time intervals.
 * Tumbling windows are inclusive start exclusive end.
 * By default the beginning instant of fist window is Epoch 0 Thu Jan 01 
00:00:00 1970 UTC.
 * Optionally users may provide a different origin as a timestamp arg3.
 *
 * This an example of series of window with an interval of 5 seconds and origin 
Epoch 0 Thu Jan 01 00:00:00 1970 UTC:
 *
 *
 *   interval 1   interval 2interval 3
 *   Jan 01 00:00:00  Jan 01 00:00:05   Jan 01 00:00:10
 * 0 -- 4 : 5 --- 9: 10 --- 14
 *
 * This UDF rounds timestamp agr1 to the beginning of window interval where it 
belongs to.
 *
 */
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints

2018-10-17 Thread Jesus Camacho Rodriguez (JIRA)
Jesus Camacho Rodriguez created HIVE-20767:
--

 Summary: Multiple project between join operators may affect join 
reordering using constraints
 Key: HIVE-20767
 URL: https://issues.apache.org/jira/browse/HIVE-20767
 Project: Hive
  Issue Type: Bug
  Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20766) support get_aggr_stats for txn stats in CachedStore

2018-10-17 Thread Sergey Shelukhin (JIRA)
Sergey Shelukhin created HIVE-20766:
---

 Summary: support get_aggr_stats for txn stats in CachedStore
 Key: HIVE-20766
 URL: https://issues.apache.org/jira/browse/HIVE-20766
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20765) validate txn stats for partitions with one call

2018-10-17 Thread Sergey Shelukhin (JIRA)
Sergey Shelukhin created HIVE-20765:
---

 Summary: validate txn stats for partitions with one call
 Key: HIVE-20765
 URL: https://issues.apache.org/jira/browse/HIVE-20765
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Review Request 69050: HIVE-20720: Add partition column option to JDBC handler

2018-10-17 Thread Jesús Camacho Rodríguez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69050/#review209714
---




jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputFormat.java
Lines 106 (patched)


I believe it is better to get lower/upper bound in a single query, hence 
making a single trip to rdbms.



jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java
Lines 233 (patched)


Could we use a case insensitive regex that would match something like 
'FROM\s+tableString'? I believe it would be safer. If you do not tackle it in 
this issue, let's create a follow-up.



jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java
Lines 241 (patched)


We should replace by "( " + boundaryQuery + ") " + tableName, since the 
outer query may reference the columns with qualified names, e.g., 
tableName.col1.


- Jesús Camacho Rodríguez


On Oct. 16, 2018, 5:08 p.m., Daniel Dai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69050/
> ---
> 
> (Updated Oct. 16, 2018, 5:08 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> See HIVE-20720.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/Constants.java 1190679 
>   itests/src/test/resources/testconfiguration.properties 9a87464 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputFormat.java 
> 74999db 
>   jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputSplit.java 
> 3a6ada8 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java 
> 1da6213 
>   jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java 
> 5947628 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java
>  18e2397 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessor.java
>  fdaa794 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java
>  abdc5f0 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java
>  a95aca2 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JethroDatabaseAccessor.java
>  db0454e 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/MySqlDatabaseAccessor.java
>  86fde7c 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DateIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DecimalIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/DoubleIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/IntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/IntervalSplitterFactory.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/LongIntervalSpitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/spitter/TimestampIntervalSplitter.java
>  PRE-CREATION 
>   
> jdbc-handler/src/test/java/org/apache/hive/storage/jdbc/TestJdbcInputFormat.java
>  b146633 
>   
> jdbc-handler/src/test/java/org/apache/hive/storage/jdbc/dao/TestGenericJdbcDatabaseAccessor.java
>  34f061e 
>   ql/src/test/queries/clientpositive/external_jdbc_table_partition.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/external_jdbc_table_typeconversion.q 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/external_jdbc_table_partition.q.out 
> PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/external_jdbc_table_typeconversion.q.out
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69050/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Daniel Dai
> 
>



[jira] [Created] (HIVE-20764) Tez: ReduceRecordProcessor::init() can avoid doing .waitForAllInputsReady()

2018-10-17 Thread Gopal V (JIRA)
Gopal V created HIVE-20764:
--

 Summary: Tez: ReduceRecordProcessor::init() can avoid doing 
.waitForAllInputsReady()
 Key: HIVE-20764
 URL: https://issues.apache.org/jira/browse/HIVE-20764
 Project: Hive
  Issue Type: Bug
Reporter: Gopal V


The ReduceRecordProcessor waits for all inputs to be ready before starting to 
initialize the input operators.

This is not necessary for the unsorted shuffle cases, as just having one input 
(the hash side) entirely ready is sufficient to start doing useful work with 
the operators (i.e build the hashtable).

{code}
if (shuffleInputs != null) {
  l4j.info("Waiting for ShuffleInputs to become ready");
  processorContext.waitForAllInputsReady(new 
ArrayList(shuffleInputs));
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20763) Add google cloud storage (gs) to the exim uri schema whitelist

2018-10-17 Thread Siddharth Seth (JIRA)
Siddharth Seth created HIVE-20763:
-

 Summary: Add google cloud storage (gs) to the exim uri schema 
whitelist
 Key: HIVE-20763
 URL: https://issues.apache.org/jira/browse/HIVE-20763
 Project: Hive
  Issue Type: Task
Reporter: Siddharth Seth
Assignee: Siddharth Seth
 Attachments: HIVE-20763.01.patch

import/export is enabled for s3a by default. Ideally this list should include 
other cloud storage options. This Jira adds Google Storage to the list.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Review Request 68946: HIVE-20707: Automatic MSCK REPAIR for external tables

2018-10-17 Thread j . prasanth . j

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68946/
---

(Updated Oct. 17, 2018, 7:14 p.m.)


Review request for hive, Ashutosh Chauhan and Jason Dere.


Changes
---

Added sorting to stabilize qfile test output


Bugs: HIVE-20707
https://issues.apache.org/jira/browse/HIVE-20707


Repository: hive-git


Description
---

HIVE-20707: Automatic partition management


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a458ac0 
  hbase-handler/src/test/results/positive/external_table_ppd.q.out edcbe7e 
  hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out 
1209c88 
  hbase-handler/src/test/results/positive/hbase_ddl.q.out ccd4148 
  hbase-handler/src/test/results/positive/hbase_queries.q.out eeb97f0 
  hbase-handler/src/test/results/positive/hbasestats.q.out 5a4aea9 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java
 a9d7468 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 807f159 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java 46bf088 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/CheckResult.java 0b4240f 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMetaStoreChecker.java 
598bb2e 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java cff32d3 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java bba7d6c 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 27f677e 
  
ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java
 ce2b186 
  
ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckDropPartitionsInBatches.java 
9480d38 
  ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 
a2a0583 
  ql/src/test/queries/clientpositive/msck_repair_acid.q PRE-CREATION 
  ql/src/test/queries/clientpositive/partition_discovery.q PRE-CREATION 
  ql/src/test/results/clientpositive/create_like.q.out f4a5ed5 
  ql/src/test/results/clientpositive/create_like_view.q.out 870f280 
  ql/src/test/results/clientpositive/default_file_format.q.out 0adf5ae 
  ql/src/test/results/clientpositive/druid/druidkafkamini_basic.q.out 883994c 
  ql/src/test/results/clientpositive/druid/druidmini_expressions.q.out 9c9af44 
  ql/src/test/results/clientpositive/druid_topn.q.out 179902a 
  ql/src/test/results/clientpositive/explain_locks.q.out ed7f1e8 
  ql/src/test/results/clientpositive/llap/external_table_purge.q.out 24c778e 
  ql/src/test/results/clientpositive/llap/mm_exim.q.out ee6cf06 
  ql/src/test/results/clientpositive/llap/strict_managed_tables2.q.out f3b6152 
  ql/src/test/results/clientpositive/llap/whroot_external1.q.out cac158c 
  ql/src/test/results/clientpositive/msck_repair_acid.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/msck_repair_drop.q.out 2456734 
  ql/src/test/results/clientpositive/partition_discovery.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/rename_external_partition_location.q.out 
02cd814 
  ql/src/test/results/clientpositive/repl_2_exim_basic.q.out b2bcd51 
  ql/src/test/results/clientpositive/show_create_table_alter.q.out 2c75c36 
  ql/src/test/results/clientpositive/show_create_table_partitioned.q.out 
e554a18 
  ql/src/test/results/clientpositive/show_create_table_serde.q.out 8b95c9b 
  ql/src/test/results/clientpositive/spark/stats_noscan_2.q.out 2d713a8 
  ql/src/test/results/clientpositive/stats_noscan_2.q.out 182820f 
  ql/src/test/results/clientpositive/temp_table_display_colstats_tbllvl.q.out 
2a442b4 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java
 PRE-CREATION 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java
 294dfb7 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/api/MetastoreException.java
 PRE-CREATION 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
 7b01678 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java
 8fb1fa7 
  
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java
 PRE-CREATION 
  
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java
 PRE-CREATION 
  
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MsckInfo.java
 PRE-CREATION 
  
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MsckPartitionExpressionProxy.java
 PRE-CREATION 
  
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
 b98b4b4 
  

Re: Review Request 69017: HIVE-20718

2018-10-17 Thread Jesús Camacho Rodríguez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69017/
---

(Updated Oct. 17, 2018, 6:42 p.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-20718
https://issues.apache.org/jira/browse/HIVE-20718


Repository: hive-git


Description
---

HIVE-20718


Diffs (updated)
-

  data/conf/perf-reg/tez/hive-site.xml 78a5481e0333a3ce9bc516e03273abe6a51c9a49 
  data/scripts/q_perf_test_init_constraints.sql PRE-CREATION 
  
itests/qtest/src/test/java/org/apache/hadoop/hive/cli/TestTezPerfCliDriver.java 
98ceb214047ba56fc2e1ebabc7b0860f22524203 
  
itests/qtest/src/test/java/org/apache/hadoop/hive/cli/TestTezPerfConstraintsCliDriver.java
 PRE-CREATION 
  itests/src/test/resources/testconfiguration.properties 
b6d42c64af002aa37a9c088dea8ea6b41c96c950 
  itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java 
5e1e88e89d29f94006764e09cf1c60e58cffdc54 
  ql/src/java/org/apache/hadoop/hive/ql/Context.java 
b4d5806d4ed7f23f2dcf5299fe3c0b2fbe22ff80 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java 
46bf088f2c5ca97e11cc7ab939ef8ddaefd453c6 
  ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java 
92fcfec673fd7d0cb2ce92bd6fffa3eee2b9b1da 
  ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java 
0af30d48f32f1e6a8286c869db9182ba9c5557ed 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelOptUtil.java 
dc0a84b37dc3213c52d331471cb0e16bd499886e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinConstraintsRule.java
 0a307f248aff8c72e7c52a425181a50dd5dd2023 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
22f3266c87f1d42c254893b424b68e757fb2953b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainConfiguration.java 
a92502e74646f15a68a3fd488d71570d7a068566 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSemanticAnalyzer.java 
49b614634ff9196d5ef97a300105414f857200bd 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 
8bf9cc0ad69dc96f022cde6f500dd3cd68bf9300 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
bc95c46d24098fa5706ab8178eddd0d744d4f57d 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExplainWork.java 
01da4d558d0737df0408fe5d9050641cca550e46 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestUpdateDeleteSemanticAnalyzer.java
 932f4e850b6b197b3cc67df007341f2e49900921 
  ql/src/test/queries/clientpositive/perf/cbo_query1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query10.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query11.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query12.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query13.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query14.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query15.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query16.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query17.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query18.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query19.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query20.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query21.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query22.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query23.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query24.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query25.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query26.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query27.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query28.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query29.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query30.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query31.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query32.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query33.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query34.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query35.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query36.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query37.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query38.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query39.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query4.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/cbo_query40.q PRE-CREATION 
  

Re: skip batching of tests

2018-10-17 Thread Vihang Karajgaonkar
>
> master-mr2.properties has to be updated on the ptest servers and restarted
> afaik (atleast it used to be).


Don't think we need a restart. Just updating the properties file should be
enough.

I have updated the test configuration so that these tests are skipped from
batching for now. I still think we should find a better way to to speed up
these tests since this will potentially increase the time taken for all the
jobs now. Lets see how much of time it adds to the overall time taken for
each job. If its significant, can we split these tests into smaller ones so
that we can still run them in parallel. I see the next job id in the queue
is #14526. I will keep an eye on it to see if the configuration change has
taken effect.

On Wed, Oct 17, 2018 at 3:26 AM, Peter Vary 
wrote:

> You should go to the ptest server itself and modify
> /usr/local/hiveptest/profiles/master-mr2.properties to change the
> configuration
>
> > On Oct 17, 2018, at 11:36, Prasanth Jayachandran <
> pjayachand...@hortonworks.com> wrote:
> >
> > Hi
> >
> > master-mr2.properties has to be updated on the ptest servers and
> restarted afaik (atleast it used to be).
> >
> > Thanks
> > Prasanth
> >
> >
> >
> > On Wed, Oct 17, 2018 at 2:10 AM -0700, "Anishek Agarwal" <
> anis...@apache.org> wrote:
> >
> >
> > Hello,
> >
> > There are few tests w.r.t to Replication in hive that are taking long to
> > run and hence more often than not timeout as part of the batch they form.
> > All these tests are in org.apache.hadoop.hive.ql.parse package. I tried
> to
> > skipBatching for these tests by changing the master-mr2.properties by
> > adding these test classes to the property
> > "ut.itests.hive-unit.skipBatching", however that does not seem to work
> as
> > the tests are still batched and in test on my patch i see "did not
> produce
> > a TEST-*.xml file (likely timed out) (batchId=244)"
> >
> > does anyone know how to skip batching for these tests ?
> >
> > Thanks
> > anishek
> >
>
>


[jira] [Created] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.

2018-10-17 Thread Sankar Hariappan (JIRA)
Sankar Hariappan created HIVE-20762:
---

 Summary: NOTIFICATION_LOG cleanup interval is hardcoded as 60s and 
is too small.
 Key: HIVE-20762
 URL: https://issues.apache.org/jira/browse/HIVE-20762
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, Metastore
Affects Versions: 4.0.0
Reporter: Sankar Hariappan
Assignee: Sankar Hariappan


NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It 
should be set to several hours or else the number of metastore calls would be 
too high and impact other operations.
Make it configurable item and set it as 2Hrs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20761) Select for update on notification sequence table has retry interval and retries count too small.

2018-10-17 Thread Sankar Hariappan (JIRA)
Sankar Hariappan created HIVE-20761:
---

 Summary: Select for update on notification sequence table has 
retry interval and retries count too small.
 Key: HIVE-20761
 URL: https://issues.apache.org/jira/browse/HIVE-20761
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, Metastore
Affects Versions: 4.0.0
Reporter: Sankar Hariappan
Assignee: Sankar Hariappan


Hive DDL's are intermittently failing with Error- Couldn't acquire the DB log 
notification lock because we reached the maximum # of retries: 5 retries
{code:java}
Error while processing statement: FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Couldn't acquire 
the DB log notification lock because we reached the maximum # of retries: 5 
retries. If this happens too often, then is recommended to increase the maximum 
number of retries on the hive.notification.sequence.lock.max.retries 
configuration :: Error executing SQL query "select "NEXT_EVENT_ID" from 
"NOTIFICATION_SEQUENCE" for update".)
2018-08-28 01:17:56,808|INFO|MainThread|machine.py:183 - 
run()||GUID=94e6ff4d-5db8-45eb-8654-76f546e7f0b3|java.sql.SQLException: Error 
while processing statement: FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Couldn't acquire 
the DB log notification lock because we reached the maximum # of retries: 5 
retries. If this happens too often, then is recommended to increase the maximum 
number of retries on the hive.notification.sequence.lock.max.retries 
configuration :: Error executing SQL query "select "NEXT_EVENT_ID" from 
"NOTIFICATION_SEQUENCE" for update".){code}
It seems, metastore operations are slow in this cluster and hence concurrent 
writes/DDL operations are failing to lock the row for update.
Currently, the sleep interval between retries is specified via the config 
*hive.notification.sequence.lock.retry.sleep.interval*. The default value is 
500 ms which seems to be too small. Can we set higher values for sleep interval 
and retries count,

*hive.notification.sequence.lock.retry.sleep.interval=10s*

*hive.notification.sequence.lock.max.retries=10*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20760) Reducing memory overhead due to multiple HiveConfs

2018-10-17 Thread Barnabas Maidics (JIRA)
Barnabas Maidics created HIVE-20760:
---

 Summary: Reducing memory overhead due to multiple HiveConfs
 Key: HIVE-20760
 URL: https://issues.apache.org/jira/browse/HIVE-20760
 Project: Hive
  Issue Type: Improvement
  Components: Configuration
Reporter: Barnabas Maidics
 Attachments: hiveconf_interned.html, hiveconf_original.html

The issue is that every Hive task has to load its own version of {{HiveConf}}. 
When running with a large number of cores per executor (HoS), there is a 
significant (~10%) amount of memory wasted due to this duplication. 

I looked into the problem and found a way to reduce the overhead caused by the 
multiple HiveConf objects.

I've created an implementation of Properties, somewhat similar to 
CopyOnFirstWriteProperties. CopyOnFirstWriteProperties can't be used to solve 
this problem, because it drops the interned Properties right after we add a new 
property.

So my implementation looks like this:
 * When we create a new HiveConf from an existing one (copy constructor), we 
change the properties object stored by HiveConf to the new Properties 
implementation (HiveConfProperties). We have 2 possible way to do this. Either 
we change the visibility of the properties field in the ancestor class 
(Configuration which comes from hadoop) to protected, or a simpler way is to 
just change the type using reflection.
 * HiveConfProperties instantly intern the given properties. After this, every 
time we add a new property to HiveConf, we add it to an additional Properties 
object. This way if we create multiple HiveConf with the same base properties, 
they will use the same Properties object but each session/task can add its own 
unique properties.
 * Getting a property from HiveConfProperties would look like this: (I stored 
the non-interned properties in super class)

                String property=super.getProperty(key);
                if (property == null) property= interned.getProperty(key);
                return property;

Running some tests showed that the interning works (with 50 connections to 
HiveServer2, heapdumps created after sessions are created for queries): 

Overall memory:

         original: 34,599K              interned: 20,582K

Retained memory of HiveConfs:

        original: 16,366K               interned: 10,804K

I attach the JXray reports about the heapdumps.

What are your thoughts about this solution? 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Review Request 68975: HIVE-20661: Dynamic partitions loading calls add partition for every partition 1-by-1

2018-10-17 Thread Laszlo Pinter via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68975/
---

(Updated Oct. 17, 2018, 1:44 p.m.)


Review request for hive and Peter Vary.


Repository: hive-git


Description
---

HIVE-20661: Dynamic partitions loading calls add partition for every partition 
1-by-1


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/metastore/SynchronizedMetaStoreClient.java 
0ab77e84c6222d35bcec9ce95ed02014911ef144 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
4de038913a5c9a2c199f71702b8f70ca84d0856b 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java 
dd23d7db3e70c9540e48c42eb7b9a33ed775cea6 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
 aba63f050b5b98a2aeeb0df6ff2de5e6e06761f2 


Diff: https://reviews.apache.org/r/68975/diff/4/

Changes: https://reviews.apache.org/r/68975/diff/3-4/


Testing
---


Thanks,

Laszlo Pinter



Re: skip batching of tests

2018-10-17 Thread Peter Vary
You should go to the ptest server itself and modify 
/usr/local/hiveptest/profiles/master-mr2.properties to change the configuration

> On Oct 17, 2018, at 11:36, Prasanth Jayachandran 
>  wrote:
> 
> Hi
> 
> master-mr2.properties has to be updated on the ptest servers and restarted 
> afaik (atleast it used to be).
> 
> Thanks
> Prasanth
> 
> 
> 
> On Wed, Oct 17, 2018 at 2:10 AM -0700, "Anishek Agarwal" 
> mailto:anis...@apache.org>> wrote:
> 
> 
> Hello,
> 
> There are few tests w.r.t to Replication in hive that are taking long to
> run and hence more often than not timeout as part of the batch they form.
> All these tests are in org.apache.hadoop.hive.ql.parse package. I tried to
> skipBatching for these tests by changing the master-mr2.properties by
> adding these test classes to the property
> "ut.itests.hive-unit.skipBatching", however that does not seem to work as
> the tests are still batched and in test on my patch i see "did not produce
> a TEST-*.xml file (likely timed out) (batchId=244)"
> 
> does anyone know how to skip batching for these tests ?
> 
> Thanks
> anishek
> 



Re: skip batching of tests

2018-10-17 Thread Prasanth Jayachandran
Hi

master-mr2.properties has to be updated on the ptest servers and restarted 
afaik (atleast it used to be).

Thanks
Prasanth



On Wed, Oct 17, 2018 at 2:10 AM -0700, "Anishek Agarwal" 
mailto:anis...@apache.org>> wrote:


Hello,

There are few tests w.r.t to Replication in hive that are taking long to
run and hence more often than not timeout as part of the batch they form.
All these tests are in org.apache.hadoop.hive.ql.parse package. I tried to
skipBatching for these tests by changing the master-mr2.properties by
adding these test classes to the property
"ut.itests.hive-unit.skipBatching", however that does not seem to work as
the tests are still batched and in test on my patch i see "did not produce
a TEST-*.xml file (likely timed out) (batchId=244)"

does anyone know how to skip batching for these tests ?

Thanks
anishek



[GitHub] hive pull request #318: Hive 18832

2018-10-17 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/318


---


skip batching of tests

2018-10-17 Thread Anishek Agarwal
Hello,

There are few tests w.r.t to Replication in hive that are taking long to
run and hence more often than not timeout as part of the batch they form.
All these tests are in org.apache.hadoop.hive.ql.parse package. I tried to
skipBatching for these tests by changing the master-mr2.properties by
adding these test classes to the property
"ut.itests.hive-unit.skipBatching", however that does not seem to work as
the tests are still batched and in test on my patch i see "did not produce
a TEST-*.xml file (likely timed out) (batchId=244)"

does anyone know how to skip batching for these tests ?

Thanks
anishek


[GitHub] hive pull request #448: HIVE-20751: Upgrade arrow version to 0.10.0

2018-10-17 Thread ShubhamChaurasia
GitHub user ShubhamChaurasia opened a pull request:

https://github.com/apache/hive/pull/448

HIVE-20751: Upgrade arrow version to 0.10.0

Upgraded apache arrow version from 0.8.0 to 0.10.0
Incorporated naming changes in arrow 
(https://issues.apache.org/jira/browse/ARROW-1815)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ShubhamChaurasia/hive HIVE-20751

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/448.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #448


commit 27335008d5653e55a1b24f048b568e46417b8ebd
Author: ShubhamChaurasia 
Date:   2018-10-17T05:59:53Z

HIVE-20751: Upgrade arrow version to 0.10.0




---