[jira] [Updated] (HIVE-21737) Upgrade Avro to version 1.9.1

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-21737:

Status: Open  (was: Patch Available)

> Upgrade Avro to version 1.9.1
> -
>
> Key: HIVE-21737
> URL: https://issues.apache.org/jira/browse/HIVE-21737
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Ismaël Mejía
>Assignee: Fokko Driesprong
>Priority: Major
>  Labels: pull-request-available
> Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch, 
> 0002-HIVE-21737-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner 
> version of Avro without Jackson in the public API. Worth the update.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Status: Patch Available  (was: Open)

Attaching same patch again. Can't get Findbugs to run locally and the Jenkins 
builds have expired already.

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch, HIVE-20855.2.patch, 
> HIVE-20855.3.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Attachment: HIVE-20855.3.patch

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch, HIVE-20855.2.patch, 
> HIVE-20855.3.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Status: Open  (was: Patch Available)

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch, HIVE-20855.2.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Status: Patch Available  (was: Open)

Trying again. I forgot about itests.

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch, HIVE-20855.2.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Attachment: HIVE-20855.2.patch

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch, HIVE-20855.2.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Status: Open  (was: Patch Available)

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch, HIVE-20855.2.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673282#comment-16673282
 ] 

Lars Francke commented on HIVE-20855:
-

This compiles on my end so I must have uploaded an outdated patch or done 
something different wrong. Either way I'll look at it.

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-20287) Document the differences between managed and external tables

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke resolved HIVE-20287.
-
Resolution: Fixed

> Document the differences between managed and external tables
> 
>
> Key: HIVE-20287
> URL: https://issues.apache.org/jira/browse/HIVE-20287
> Project: Hive
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Major
>
> We should document all the differences between managed and external tables. I 
> need everyone's help for that though.
> This is what I know:
> * ARCHIVE/UNARCHIVE - DDLTask - Only works for managed tables
> * TRUNCATE - DDLSemanticAnalyzer - Only works for managed tables
> * MERGE/CONCATENATE - HiveRelOpMaterializationValidator - Only works for 
> managed tables
> * Constraints - DDLSemanticAnalyzer -  (NOT NULL, DEFAULT, CHECK, only RELY 
> ist allowed)
> * IMPORT - ImportSemanticAnalyzer - This has some wild restrictions I didn't 
> follow for external tables
> * Query Results Caching - https://issues.apache.org/jira/browse/HIVE-18513 
> SemanticAnalyzer
>  
> Hortonworks has extra documentation listing these things:
> * Query cache
> * Materialized views, except in a limited way
> * Default statistics gathering
> * Compute queries using statistics
> * Automatic runtime filtering
> * File merging after insert
>  
> It'd be great if someone (from Hortonworks or otherwise) could elaborate on 
> those.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20287) Document the differences between managed and external tables

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673028#comment-16673028
 ] 

Lars Francke commented on HIVE-20287:
-

I have created 
https://cwiki.apache.org/confluence/display/Hive/Managed+vs.+External+Tables

Any additions are still welcome but I'll close this issue as fixed for now.

> Document the differences between managed and external tables
> 
>
> Key: HIVE-20287
> URL: https://issues.apache.org/jira/browse/HIVE-20287
> Project: Hive
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Major
>
> We should document all the differences between managed and external tables. I 
> need everyone's help for that though.
> This is what I know:
> * ARCHIVE/UNARCHIVE - DDLTask - Only works for managed tables
> * TRUNCATE - DDLSemanticAnalyzer - Only works for managed tables
> * MERGE/CONCATENATE - HiveRelOpMaterializationValidator - Only works for 
> managed tables
> * Constraints - DDLSemanticAnalyzer -  (NOT NULL, DEFAULT, CHECK, only RELY 
> ist allowed)
> * IMPORT - ImportSemanticAnalyzer - This has some wild restrictions I didn't 
> follow for external tables
> * Query Results Caching - https://issues.apache.org/jira/browse/HIVE-18513 
> SemanticAnalyzer
>  
> Hortonworks has extra documentation listing these things:
> * Query cache
> * Materialized views, except in a limited way
> * Default statistics gathering
> * Compute queries using statistics
> * Automatic runtime filtering
> * File merging after insert
>  
> It'd be great if someone (from Hortonworks or otherwise) could elaborate on 
> those.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Attachment: HIVE-20855.1.patch

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Attachment: (was: HIVE-20855.1.patch)

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Attachment: HIVE-20855.1.patch

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-20855:

Status: Patch Available  (was: Open)

> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-16520) Cache hive metadata in metastore

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672072#comment-16672072
 ] 

Lars Francke commented on HIVE-16520:
-

I added documentation to the Configuration Properties site and this one 
https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+3.0+Administration

> Cache hive metadata in metastore
> 
>
> Key: HIVE-16520
> URL: https://issues.apache.org/jira/browse/HIVE-16520
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-16520-1.patch, HIVE-16520-proto-2.patch, 
> HIVE-16520-proto.patch, HIVE-16520.2.patch, HIVE-16520.3.patch, 
> HIVE-16520.4.patch
>
>
> During Hive 2 benchmark, we find Hive metastore operation take a lot of time 
> and thus slow down Hive compilation. In some extreme case, it takes much 
> longer than the actual query run time. Especially, we find the latency of 
> cloud db is very high and 90% of total query runtime is waiting for metastore 
> SQL database operations. Based on this observation, the metastore operation 
> performance will be greatly enhanced if we have a memory structure which 
> cache the database query result.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke reassigned HIVE-20855:
---


> Clean up checkstyle & other warnings for CachedStore
> 
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings, 
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three 
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for 
> feedback on the mailing list and will commit after a week or so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18264) CachedStore: Store cached partitions/col stats within the table cache and make prewarm non-blocking

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671334#comment-16671334
 ] 

Lars Francke commented on HIVE-18264:
-

[~vgumashta] could you please update the Fix Version of this issue? Is it part 
of 3.0.0?

> CachedStore: Store cached partitions/col stats within the table cache and 
> make prewarm non-blocking
> ---
>
> Key: HIVE-18264
> URL: https://issues.apache.org/jira/browse/HIVE-18264
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-18264.1.patch, HIVE-18264.2.patch, 
> HIVE-18264.3.patch, HIVE-18264.4.patch, HIVE-18264.5.patch, 
> HIVE-18264.6.patch, HIVE-18264.7.patch, HIVE-18264.8.patch, HIVE-18264.8.patch
>
>
> Currently we have a separate cache for partitions and partition col stats 
> which results in some calls iterating through each of these for 
> retrieving/updating. For example, to modify a partition col stat, currently 
> we need to lock table, partition and partition col stats caches which are all 
> separate hashmaps. We can get better performance by organizing 
> hierarchically. For example, we can have a partition, partition col stats and 
> table col stats cache per table to improve on the previous mechanisms. This 
> will also result in better concurrency, since now instead of locking the 
> whole cache, we can selectively lock the table cache and modify multiple 
> tables in parallel. 
> In addition, currently, the prewarm mechanism populates all the caches 
> initially (it skips tables that do not pass whitelist/blacklist filter) and 
> it is a blocking call. This patch also makes prewarm non-blocking so that the 
> calls for tables that are already cached can be served from the memory and 
> the ones that are not can be served from the rdbms. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20287) Document the differences between managed and external tables

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597324#comment-16597324
 ] 

Lars Francke commented on HIVE-20287:
-

The docs currently say:

??An external table describes the metadata / schema on external files. External 
table files can be accessed and managed by processes outside of Hive. External 
tables can access data stored in sources such as Azure Storage Volumes (ASV) or 
remote HDFS locations. If the structure or partitioning of an external table is 
changed, an [MSCK REPAIR TABLE 
table_name|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RecoverPartitions(MSCKREPAIRTABLE)]
 statement can be used to refresh metadata information.??

This makes it sound as if MSCK only works for external tables. Is that true? 
I'd need to check. Is anyone certain?

> Document the differences between managed and external tables
> 
>
> Key: HIVE-20287
> URL: https://issues.apache.org/jira/browse/HIVE-20287
> Project: Hive
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Major
>
> We should document all the differences between managed and external tables. I 
> need everyone's help for that though.
> This is what I know:
> * ARCHIVE/UNARCHIVE - DDLTask - Only works for managed tables
> * TRUNCATE - DDLSemanticAnalyzer - Only works for managed tables
> * MERGE/CONCATENATE - HiveRelOpMaterializationValidator - Only works for 
> managed tables
> * Constraints - DDLSemanticAnalyzer -  (NOT NULL, DEFAULT, CHECK, only RELY 
> ist allowed)
> * IMPORT - ImportSemanticAnalyzer - This has some wild restrictions I didn't 
> follow for external tables
> * Query Results Caching - https://issues.apache.org/jira/browse/HIVE-18513 
> SemanticAnalyzer
>  
> Hortonworks has extra documentation listing these things:
> * Query cache
> * Materialized views, except in a limited way
> * Default statistics gathering
> * Compute queries using statistics
> * Automatic runtime filtering
> * File merging after insert
>  
> It'd be great if someone (from Hortonworks or otherwise) could elaborate on 
> those.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20287) Document the differences between managed and external tables

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597318#comment-16597318
 ] 

Lars Francke commented on HIVE-20287:
-

Thanks Eugene!

> Document the differences between managed and external tables
> 
>
> Key: HIVE-20287
> URL: https://issues.apache.org/jira/browse/HIVE-20287
> Project: Hive
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Major
>
> We should document all the differences between managed and external tables. I 
> need everyone's help for that though.
> This is what I know:
> * ARCHIVE/UNARCHIVE - DDLTask - Only works for managed tables
> * TRUNCATE - DDLSemanticAnalyzer - Only works for managed tables
> * MERGE/CONCATENATE - HiveRelOpMaterializationValidator - Only works for 
> managed tables
> * Constraints - DDLSemanticAnalyzer -  (NOT NULL, DEFAULT, CHECK, only RELY 
> ist allowed)
> * IMPORT - ImportSemanticAnalyzer - This has some wild restrictions I didn't 
> follow for external tables
> * Query Results Caching - https://issues.apache.org/jira/browse/HIVE-18513 
> SemanticAnalyzer
>  
> Hortonworks has extra documentation listing these things:
> * Query cache
> * Materialized views, except in a limited way
> * Default statistics gathering
> * Compute queries using statistics
> * Automatic runtime filtering
> * File merging after insert
>  
> It'd be great if someone (from Hortonworks or otherwise) could elaborate on 
> those.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase KeyValue

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-2828:
---
Labels:   (was: TODOC15)

> make timestamp accessible in the hbase KeyValue 
> 
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 1.1.0
>
> Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.1.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.2.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.3.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.4.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.5.patch, HIVE-2828.6.patch.txt, 
> HIVE-2828.7.patch.txt, HIVE-2828.8.patch.txt, HIVE-2828.9.patch.txt
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful 
> to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user 
> might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
>   STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>   WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-2828) make timestamp accessible in the hbase KeyValue

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581125#comment-16581125
 ] 

Lars Francke commented on HIVE-2828:


I added a note in the docs about this

> make timestamp accessible in the hbase KeyValue 
> 
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
>  Labels: TODOC15
> Fix For: 1.1.0
>
> Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.1.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.2.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.3.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.4.patch, 
> ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.5.patch, HIVE-2828.6.patch.txt, 
> HIVE-2828.7.patch.txt, HIVE-2828.8.patch.txt, HIVE-2828.9.patch.txt
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful 
> to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user 
> might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
>   STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>   WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18513) Query results caching

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565097#comment-16565097
 ] 

Lars Francke commented on HIVE-18513:
-

Thanks Jason. If that's all that needs changing I can try to provide a patch as 
well.

 

Can we also get some documentation around this feature? I cannot find any words 
on memory management for example. Where are results stored/kept?

> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch, 
> HIVE-18513.3.patch, HIVE-18513.4.patch, HIVE-18513.5.patch, HIVE-18513.6.patch
>
>
> Add a query results cache that can save the results of an executed Hive query 
> for reuse on subsequent queries. This may be useful in cases where the same 
> query is issued many times, since Hive can return back the results of a 
> cached query rather than having to execute the full query on the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-20287) Document the differences between managed and external tables

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke reassigned HIVE-20287:
---


> Document the differences between managed and external tables
> 
>
> Key: HIVE-20287
> URL: https://issues.apache.org/jira/browse/HIVE-20287
> Project: Hive
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Major
>
> We should document all the differences between managed and external tables. I 
> need everyone's help for that though.
> This is what I know:
> * ARCHIVE/UNARCHIVE - DDLTask - Only works for managed tables
> * TRUNCATE - DDLSemanticAnalyzer - Only works for managed tables
> * MERGE/CONCATENATE - HiveRelOpMaterializationValidator - Only works for 
> managed tables
> * Constraints - DDLSemanticAnalyzer -  (NOT NULL, DEFAULT, CHECK, only RELY 
> ist allowed)
> * IMPORT - ImportSemanticAnalyzer - This has some wild restrictions I didn't 
> follow for external tables
> * Query Results Caching - https://issues.apache.org/jira/browse/HIVE-18513 
> SemanticAnalyzer
>  
> Hortonworks has extra documentation listing these things:
> * Query cache
> * Materialized views, except in a limited way
> * Default statistics gathering
> * Compute queries using statistics
> * Automatic runtime filtering
> * File merging after insert
>  
> It'd be great if someone (from Hortonworks or otherwise) could elaborate on 
> those.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HIVE-18513) Query results caching

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549661#comment-16549661
 ] 

Lars Francke edited comment on HIVE-18513 at 7/19/18 6:35 PM:
--

If I read the design doc correctly the decision to disallow EXTERNAL tables is 
not a purely technical one but more one of expectations. EXTERNAL tables are 
expected to be modified outside of Hive's control while MANAGED tables are not.

I agree with [~thai.bui] that it'd be great to give the user the choice to 
still use this feature. In my experience both MANAGED and EXTERNAL tables are 
frequently modified outside of Hive (ingestion directly via Sqoop, Flume, NiFi, 
Spark etc.). Have you made any progress Thai Bui?

 

(This comment is not intended as criticism, I understand the reasoning. It is 
for documentation purposes)


was (Author: lars_francke):
If I read the design doc correctly the decision to disallow EXTERNAL tables is 
not a purely technical one but more one of expectations. EXTERNAL tables are 
expected to be modified outside of Hive's control while MANAGED tables are not.

I agree with [~thai.bui] that it'd be great to give the user the choice to 
still use this feature. In my experience both MANAGED and EXTERNAL tables are 
frequently modified outside of Hive (ingestion directly via Sqoop, Flume, NiFi, 
Spark etc.). Have you made any progress Thai Bui?

> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch, 
> HIVE-18513.3.patch, HIVE-18513.4.patch, HIVE-18513.5.patch, HIVE-18513.6.patch
>
>
> Add a query results cache that can save the results of an executed Hive query 
> for reuse on subsequent queries. This may be useful in cases where the same 
> query is issued many times, since Hive can return back the results of a 
> cached query rather than having to execute the full query on the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18513) Query results caching

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549661#comment-16549661
 ] 

Lars Francke commented on HIVE-18513:
-

If I read the design doc correctly the decision to disallow EXTERNAL tables is 
not a purely technical one but more one of expectations. EXTERNAL tables are 
expected to be modified outside of Hive's control while MANAGED tables are not.

I agree with [~thai.bui] that it'd be great to give the user the choice to 
still use this feature. In my experience both MANAGED and EXTERNAL tables are 
frequently modified outside of Hive (ingestion directly via Sqoop, Flume, NiFi, 
Spark etc.). Have you made any progress Thai Bui?

> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch, 
> HIVE-18513.3.patch, HIVE-18513.4.patch, HIVE-18513.5.patch, HIVE-18513.6.patch
>
>
> Add a query results cache that can save the results of an executed Hive query 
> for reuse on subsequent queries. This may be useful in cases where the same 
> query is issued many times, since Hive can return back the results of a 
> cached query rather than having to execute the full query on the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18712) Design HMS Api v2

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409462#comment-16409462
 ] 

Lars Francke commented on HIVE-18712:
-

gRPC is great but it does not support Kerberos authentication and I assume we 
won't get away from Kerberos anytime soon. We'd need to do custom code (sending 
a ticket as metadata) etc. also encryption would only really be possible using 
TLS and not on the Kerberos/SASL layer. I'm not _against_ gRPC but it's 
something we'd need to think about.

> Design HMS Api v2
> -
>
> Key: HIVE-18712
> URL: https://issues.apache.org/jira/browse/HIVE-18712
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
>
> This is an umbrella Jira covering the design of Hive Metastore API v2.
> It is supposed to be a placeholder for discussion and design documents.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-15434) Add UDF to allow interrogation of uniontype values

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824534#comment-15824534
 ] 

Lars Francke commented on HIVE-15434:
-

Thanks [~teabot]. Yeah I think I did document some of this in the past but that 
was more or less a coincidence as I stumbled across the feature when I needed 
it. I'm afraid I currently won't be able to help here.

> Add UDF to allow interrogation of uniontype values
> --
>
> Key: HIVE-15434
> URL: https://issues.apache.org/jira/browse/HIVE-15434
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Affects Versions: 2.1.1
>Reporter: David Maughan
>Assignee: David Maughan
> Attachments: HIVE-15434.01.patch, HIVE-15434.02.patch
>
>
> h2. Overview
> As stated in the documention:
> {quote}
> UNIONTYPE support is incomplete The UNIONTYPE datatype was introduced in Hive 
> 0.7.0 (HIVE-537), but full support for this type in Hive remains incomplete. 
> Queries that reference UNIONTYPE fields in JOIN (HIVE-2508), WHERE, and GROUP 
> BY clauses will fail, and Hive does not define syntax to extract the tag or 
> value fields of a UNIONTYPE. This means that UNIONTYPEs are effectively 
> look-at-only.
> {quote}
> It is essential to have a usable uniontype. Until full support is added to 
> Hive users should at least have the ability to inspect and extract values for 
> further comparison or transformation.
> h2. Proposal
> I propose to add a GenericUDF that has 2 modes of operation. Consider the 
> following schema and data that contains a union:
> Schema:
> {code}
> struct>
> {code}
> Query:
> {code}
> hive> select field1 from thing;
> {0:0}
> {1:"one"}
> {code}
> h4. Explode to Struct
> This method will recursively convert all unions within the type to structs 
> with fields named {{tag_n}}, {{n}} being the tag number. Only the {{tag_*}} 
> field that matches the tag of the union will be populated with the value. In 
> the case above the schema of field1 will be converted to:
> {code}
> struct
> {code}
> {code}
> hive> select extract_union(field1) from thing;
> {"tag_0":0,"tag_1":null}
> {"tag_0":null,"tag_1":one}
> {code}
> {code}
> hive> select extract_union(field1).tag_0 from thing;
> 0
> null
> {code}
> h4. Extract the specified tag
> This method will simply extract the value of the specified tag. If the tag 
> number matches then the value is returned, if it does not, then null is 
> returned.
> {code}
> hive> select extract_union(field1, 0) from thing;
> 0
> null
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9298) Support reading alternate timestamp formats

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451750#comment-15451750
 ] 

Lars Francke commented on HIVE-9298:


[~jdere] it'd be great if you could document this as it's a very useful feature

> Support reading alternate timestamp formats
> ---
>
> Key: HIVE-9298
> URL: https://issues.apache.org/jira/browse/HIVE-9298
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Reporter: Jason Dere
>Assignee: Jason Dere
>  Labels: TODOC1.2
> Fix For: 1.2.0
>
> Attachments: HIVE-9298.1.patch, HIVE-9298.2.patch, HIVE-9298.3.patch
>
>
> There are some users who want to be able to parse ISO-8601 timestamps, as 
> well to set their own custom timestamp formats. We may be able to support 
> this in LazySimpleSerDe through the use of a SerDe parameter to specify one 
> or more alternative timestamp patterns to use to parse timestamp values from 
> string.
> If we are doing this it might also be nice to work in support for HIVE-3844, 
> to parse numeric strings as timestamp by treating the numeric value as millis 
> since Unix epoch. This can be enabled through the SerDe params as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12671) Add documentation for HS2 service discovery

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-12671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358737#comment-15358737
 ] 

Lars Francke commented on HIVE-12671:
-

Hey [~vgumashta] would you mind fixing this? All the other stuff has been 
committed but documentation is still missing.

> Add documentation for HS2 service discovery 
> 
>
> Key: HIVE-12671
> URL: https://issues.apache.org/jira/browse/HIVE-12671
> Project: Hive
>  Issue Type: Sub-task
>  Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12467) Add number of dynamic partitions to error message

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298063#comment-15298063
 ] 

Lars Francke commented on HIVE-12467:
-

Thanks [~prasanth_j]. There's so much stuff I do not know. Today I learned 
about the q.out files. It looks like none of them are affected and the test 
failures seem unrelated to me. From all I can see this looks good to go?

> Add number of dynamic partitions to error message
> -
>
> Key: HIVE-12467
> URL: https://issues.apache.org/jira/browse/HIVE-12467
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-12467.2.patch, HIVE-12467.patch
>
>
> Currently when using dynamic partition insert we get an error message saying 
> that the client tried to create too many dynamic partitions ("Maximum was set 
> to"). I'll extend the error message to specify the number of dynamic 
> partitions which can be helpful for debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-13539) HiveHFileOutputFormat searching the wrong directory for HFiles?

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-13539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-13539:

Assignee: Tim Robertson  (was: Sushanth Sowmyan)

> HiveHFileOutputFormat searching the wrong directory for HFiles?
> ---
>
> Key: HIVE-13539
> URL: https://issues.apache.org/jira/browse/HIVE-13539
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 1.1.0
> Environment: Built into CDH 5.4.7
>Reporter: Tim Robertson
>Assignee: Tim Robertson
>Priority: Blocker
>
> When creating HFiles for a bulkload in HBase I believe it is looking in the 
> wrong directory to find the HFiles, resulting in the following exception:
> {code}
> Error: java.lang.RuntimeException: Hive Runtime Error while closing 
> operators: java.io.IOException: Multiple family directories found in 
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
>   at 
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:295)
>   at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
>   at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
>   at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
>   at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.io.IOException: Multiple family directories found in 
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:188)
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:958)
>   at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:598)
>   at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
>   at 
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
>   ... 7 more
> Caused by: java.io.IOException: Multiple family directories found in 
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
>   at 
> org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:158)
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:185)
>   ... 11 more
> {code}
> The issue is that is looks for the HFiles in 
> {{hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary}}
>  when I believe it should be looking in the task attempt subfolder, such as 
> {{hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary/attempt_1461004169450_0002_r_00_1000}}.
> This can be reproduced in any HFile creation such as:
> {code:sql}
> CREATE TABLE coords_hbase(id INT, x DOUBLE, y DOUBLE)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
>   'hbase.columns.mapping' = ':key,o:x,o:y',
>   'hbase.table.default.storage.type' = 'binary');
> SET hfile.family.path=/tmp/coords_hfiles/o; 
> SET hive.hbase.generatehfiles=true;
> INSERT OVERWRITE TABLE coords_hbase 
> SELECT id, decimalLongitude, decimalLatitude
> FROM source
> CLUSTER BY id; 
> {code}
> Any advice greatly appreciated



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-12467) Add number of dynamic partitions to error message

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-12467:

Attachment: HIVE-12467.2.patch

Thanks for taking a look Prasanth.

I have attached a new version with the suggested changes. And will commit in a 
couple of days if there are no more objections.

> Add number of dynamic partitions to error message
> -
>
> Key: HIVE-12467
> URL: https://issues.apache.org/jira/browse/HIVE-12467
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-12467.2.patch, HIVE-12467.patch
>
>
> Currently when using dynamic partition insert we get an error message saying 
> that the client tried to create too many dynamic partitions ("Maximum was set 
> to"). I'll extend the error message to specify the number of dynamic 
> partitions which can be helpful for debugging.
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-12467) Add number of dynamic partitions to error message

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-12467:

Status: Open  (was: Patch Available)

> Add number of dynamic partitions to error message
> -
>
> Key: HIVE-12467
> URL: https://issues.apache.org/jira/browse/HIVE-12467
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-12467.patch
>
>
> Currently when using dynamic partition insert we get an error message saying 
> that the client tried to create too many dynamic partitions ("Maximum was set 
> to"). I'll extend the error message to specify the number of dynamic 
> partitions which can be helpful for debugging.
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-12467) Add number of dynamic partitions to error message

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-12467:

Description: 
Currently when using dynamic partition insert we get an error message saying 
that the client tried to create too many dynamic partitions ("Maximum was set 
to"). I'll extend the error message to specify the number of dynamic partitions 
which can be helpful for debugging.

NO PRECOMMIT TESTS

  was:Currently when using dynamic partition insert we get an error message 
saying that the client tried to create too many dynamic partitions ("Maximum 
was set to"). I'll extend the error message to specify the number of dynamic 
partitions which can be helpful for debugging.


> Add number of dynamic partitions to error message
> -
>
> Key: HIVE-12467
> URL: https://issues.apache.org/jira/browse/HIVE-12467
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-12467.patch
>
>
> Currently when using dynamic partition insert we get an error message saying 
> that the client tried to create too many dynamic partitions ("Maximum was set 
> to"). I'll extend the error message to specify the number of dynamic 
> partitions which can be helpful for debugging.
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-12693) LLAP: Use Slider Anti-Affinity scheduling mode for daemon distribution

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-12693:

Release Note:   (was: commit 03473cc7fb830821b6d90f465cb604550388097d
Author: Gopal V 

HIVE-12693: LLAP: Use Slider Anti-Affinity scheduling mode for daemon 
distribution (Gopal V, reviewed by Sergey Shelukhin)

Signed-off-by: Gopal V )

> LLAP: Use Slider Anti-Affinity scheduling mode for daemon distribution
> --
>
> Key: HIVE-12693
> URL: https://issues.apache.org/jira/browse/HIVE-12693
> Project: Hive
>  Issue Type: Improvement
>  Components: llap
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
> Fix For: 2.0.0, 2.1.0
>
> Attachments: HIVE-12693.1.patch, HIVE-12693.2.patch
>
>
> Slider has SLIDER-82 which adds anti-affinity placement policies for 
> containers, to avoid colliding on to the same machine when deploying LLAP 
> instances.
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-6972) jdbc HTTP configuration options should be part of sessionConf part of connection string

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046947#comment-15046947
 ] 

Lars Francke commented on HIVE-6972:


I just documented this in the wiki on both pages [~leftylev] mentioned.

> jdbc HTTP configuration options should be part of sessionConf part of 
> connection string
> ---
>
> Key: HIVE-6972
> URL: https://issues.apache.org/jira/browse/HIVE-6972
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, JDBC
>Affects Versions: 0.13.0
>Reporter: Thejas M Nair
>Assignee: Vaibhav Gumashta
> Fix For: 0.14.0
>
> Attachments: HIVE-6972.1.patch, HIVE-6972.2.patch, HIVE-6972.3.patch, 
> HIVE-6972.4.patch
>
>
> The http connection parameters are currently part of the HiveConf section of 
> the connection string. It should ideally be part of SessionConf section, as 
> that is where rest of the connection parameters are.
> HTTP transport parameters are not part of the hiveconfiguration parameters 
> that need to be set.
> ie
> instead of 
> jdbc:hive2://:/?hive.server2.transport.mode=http;hive.server2.thrift.http.path=
> it should be -
> jdbc:hive2://:/;transportMode=http;httpPath=
> *Release Note:*
> The JDBC Uri for HiveServer2 has the form:
> {code}
> jdbc:hive2://host1:port1,host2:port2/dbName;sess_var_list?hive_conf_list#hive_var_list
> {code}
> Ideally, sess_var_list is supposed to be used to pass parameters that will be 
> used within the JDBC Driver. However, some of the http mode parameters were 
> being passed in the hive_conf_list which we've fixed now (in a backward 
> compatible way).
> Additionally, JDBC Driver has a naming convention for the parameters in the 
> JDBC Uri which were not followed to the sasl.qop parameter. We've renamed it 
> in a backward compatible way.
> Details:
> *1. HTTP Mode params:*
> *Old example uri:*
> {code}
> jdbc:hive2://host:port/dbName;user=username;password=password?hive.server2.transport.mode=http;hive.server2.thrift.http.path=httpEndpoint
> {code}
> *New example uri:*
> {code}
> jdbc:hive2://host:port/dbName;user=username;password=password;transportMode=http;httpPath=httpEndpoint?hive.server2.logging.operation.enabled=false
> {code}
> As you can see above, {{hive.server2.transport.mode}} has been renamed to 
> {{transportMode}}, {{hive.server2.thrift.http.path}} has been renamed to 
> {{httpPath}} and both have been moved from {{hive_conf_list}} to 
> {{sess_var_list}}.
> *2. Sasl qop renaming:*
> *Old example uri:*
> {code}
> jdbc:hive2://host:port/dbName;principal=hiveserver2KerberosPrincipal;sasl.qop=qopValue
> {code}
> *New example uri:*
> {code}
> jdbc:hive2://host:port/dbName;principal=hiveserver2KerberosPrincipal;saslQop=qopValue
> {code}
> As you can see {{sasl.qop}} has been renamed to {{saslQop}}.
> Note: all changes are backward compatible and print a deprecation message 
> like the following:
> {code}
> 0: jdbc:hive2://localhost:10001> !connect 
> jdbc:hive2://localhost:10001/;sasl.qop=auth?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
>  username password org.apache.hive.jdbc.HiveDriver
> Connecting to 
> jdbc:hive2://localhost:10007/;sasl.qop=auth?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
> 14/10/07 16:22:24 INFO jdbc.Utils: Supplied authorities: localhost:10001
> 14/10/07 16:22:24 WARN jdbc.Utils: * JDBC param deprecation *
> 14/10/07 16:22:24 WARN jdbc.Utils: The use of sasl.qop is deprecated.
> 14/10/07 16:22:24 WARN jdbc.Utils: Please use saslQop like so: 
> jdbc:hive2://:/dbName;saslQop=
> 14/10/07 16:22:24 WARN jdbc.Utils: * JDBC param deprecation *
> 14/10/07 16:22:24 WARN jdbc.Utils: The use of hive.server2.transport.mode is 
> deprecated.
> 14/10/07 16:22:24 WARN jdbc.Utils: Please use transportMode like so: 
> jdbc:hive2://:/dbName;transportMode=
> 14/10/07 16:22:24 WARN jdbc.Utils: * JDBC param deprecation *
> 14/10/07 16:22:24 WARN jdbc.Utils: The use of hive.server2.thrift.http.path 
> is deprecated.
> 14/10/07 16:22:24 WARN jdbc.Utils: Please use httpPath like so: 
> jdbc:hive2://:/dbName;httpPath=
> Connected to: Apache Hive (version 0.14.0-SNAPSHOT)
> Driver: Hive JDBC (version 0.14.0-SNAPSHOT)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-6972) jdbc HTTP configuration options should be part of sessionConf part of connection string

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-6972:
---
Labels:   (was: TODOC14)

> jdbc HTTP configuration options should be part of sessionConf part of 
> connection string
> ---
>
> Key: HIVE-6972
> URL: https://issues.apache.org/jira/browse/HIVE-6972
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, JDBC
>Affects Versions: 0.13.0
>Reporter: Thejas M Nair
>Assignee: Vaibhav Gumashta
> Fix For: 0.14.0
>
> Attachments: HIVE-6972.1.patch, HIVE-6972.2.patch, HIVE-6972.3.patch, 
> HIVE-6972.4.patch
>
>
> The http connection parameters are currently part of the HiveConf section of 
> the connection string. It should ideally be part of SessionConf section, as 
> that is where rest of the connection parameters are.
> HTTP transport parameters are not part of the hiveconfiguration parameters 
> that need to be set.
> ie
> instead of 
> jdbc:hive2://:/?hive.server2.transport.mode=http;hive.server2.thrift.http.path=
> it should be -
> jdbc:hive2://:/;transportMode=http;httpPath=
> *Release Note:*
> The JDBC Uri for HiveServer2 has the form:
> {code}
> jdbc:hive2://host1:port1,host2:port2/dbName;sess_var_list?hive_conf_list#hive_var_list
> {code}
> Ideally, sess_var_list is supposed to be used to pass parameters that will be 
> used within the JDBC Driver. However, some of the http mode parameters were 
> being passed in the hive_conf_list which we've fixed now (in a backward 
> compatible way).
> Additionally, JDBC Driver has a naming convention for the parameters in the 
> JDBC Uri which were not followed to the sasl.qop parameter. We've renamed it 
> in a backward compatible way.
> Details:
> *1. HTTP Mode params:*
> *Old example uri:*
> {code}
> jdbc:hive2://host:port/dbName;user=username;password=password?hive.server2.transport.mode=http;hive.server2.thrift.http.path=httpEndpoint
> {code}
> *New example uri:*
> {code}
> jdbc:hive2://host:port/dbName;user=username;password=password;transportMode=http;httpPath=httpEndpoint?hive.server2.logging.operation.enabled=false
> {code}
> As you can see above, {{hive.server2.transport.mode}} has been renamed to 
> {{transportMode}}, {{hive.server2.thrift.http.path}} has been renamed to 
> {{httpPath}} and both have been moved from {{hive_conf_list}} to 
> {{sess_var_list}}.
> *2. Sasl qop renaming:*
> *Old example uri:*
> {code}
> jdbc:hive2://host:port/dbName;principal=hiveserver2KerberosPrincipal;sasl.qop=qopValue
> {code}
> *New example uri:*
> {code}
> jdbc:hive2://host:port/dbName;principal=hiveserver2KerberosPrincipal;saslQop=qopValue
> {code}
> As you can see {{sasl.qop}} has been renamed to {{saslQop}}.
> Note: all changes are backward compatible and print a deprecation message 
> like the following:
> {code}
> 0: jdbc:hive2://localhost:10001> !connect 
> jdbc:hive2://localhost:10001/;sasl.qop=auth?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
>  username password org.apache.hive.jdbc.HiveDriver
> Connecting to 
> jdbc:hive2://localhost:10007/;sasl.qop=auth?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
> 14/10/07 16:22:24 INFO jdbc.Utils: Supplied authorities: localhost:10001
> 14/10/07 16:22:24 WARN jdbc.Utils: * JDBC param deprecation *
> 14/10/07 16:22:24 WARN jdbc.Utils: The use of sasl.qop is deprecated.
> 14/10/07 16:22:24 WARN jdbc.Utils: Please use saslQop like so: 
> jdbc:hive2://:/dbName;saslQop=
> 14/10/07 16:22:24 WARN jdbc.Utils: * JDBC param deprecation *
> 14/10/07 16:22:24 WARN jdbc.Utils: The use of hive.server2.transport.mode is 
> deprecated.
> 14/10/07 16:22:24 WARN jdbc.Utils: Please use transportMode like so: 
> jdbc:hive2://:/dbName;transportMode=
> 14/10/07 16:22:24 WARN jdbc.Utils: * JDBC param deprecation *
> 14/10/07 16:22:24 WARN jdbc.Utils: The use of hive.server2.thrift.http.path 
> is deprecated.
> 14/10/07 16:22:24 WARN jdbc.Utils: Please use httpPath like so: 
> jdbc:hive2://:/dbName;httpPath=
> Connected to: Apache Hive (version 0.14.0-SNAPSHOT)
> Driver: Hive JDBC (version 0.14.0-SNAPSHOT)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-12467) Add number of dynamic partitions to error message

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-12467:

Attachment: HIVE-12467.patch

> Add number of dynamic partitions to error message
> -
>
> Key: HIVE-12467
> URL: https://issues.apache.org/jira/browse/HIVE-12467
> Project: Hive
>  Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-12467.patch
>
>
> Currently when using dynamic partition insert we get an error message saying 
> that the client tried to create too many dynamic partitions ("Maximum was set 
> to"). I'll extend the error message to specify the number of dynamic 
> partitions which can be helpful for debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8065) Support HDFS encryption functionality on Hive

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989592#comment-14989592
 ] 

Lars Francke commented on HIVE-8065:


What's the status of this? Think we can mark it as resolved? The 
encryption-branch has been merged for version 1.1.0 and I think it's confusing 
that this is still listed as unresolved.

> Support HDFS encryption functionality on Hive
> -
>
> Key: HIVE-8065
> URL: https://issues.apache.org/jira/browse/HIVE-8065
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.13.1
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>
> The new encryption support on HDFS makes Hive incompatible and unusable when 
> this feature is used.
> HDFS encryption is designed so that an user can configure different 
> encryption zones (or directories) for multi-tenant environments. An 
> encryption zone has an exclusive encryption key, such as AES-128 or AES-256. 
> Because of security compliance, the HDFS does not allow to move/rename files 
> between encryption zones. Renames are allowed only inside the same encryption 
> zone. A copy is allowed between encryption zones.
> See HDFS-6134 for more details about HDFS encryption design.
> Hive currently uses a scratch directory (like /tmp/$user/$random). This 
> scratch directory is used for the output of intermediate data (between MR 
> jobs) and for the final output of the hive query which is later moved to the 
> table directory location.
> If Hive tables are in different encryption zones than the scratch directory, 
> then Hive won't be able to renames those files/directories, and it will make 
> Hive unusable.
> To handle this problem, we can change the scratch directory of the 
> query/statement to be inside the same encryption zone of the table directory 
> location. This way, the renaming process will be successful. 
> Also, for statements that move files between encryption zones (i.e. LOAD 
> DATA), a copy may be executed instead of a rename. This will cause an 
> overhead when copying large data files, but it won't break the encryption on 
> Hive.
> Another security thing to consider is when using joins selects. If Hive joins 
> different tables with different encryption key strengths, then the results of 
> the select might break the security compliance of the tables. Let's say two 
> tables with 128 bits and 256 bits encryption are joined, then the temporary 
> results might be stored in the 128 bits encryption zone. This will conflict 
> with the table encrypted with 256 bits temporary.
> To fix this, Hive should be able to select the scratch directory that is more 
> secured/encrypted in order to save the intermediate data temporary with no 
> compliance issues.
> For instance:
> {noformat}
> SELECT * FROM table-aes128 t1 JOIN table-aes256 t2 WHERE t1.id == t2.id;
> {noformat}
> - This should use a scratch directory (or staging directory) inside the 
> table-aes256 table location.
> {noformat}
> INSERT OVERWRITE TABLE table-unencrypted SELECT * FROM table-aes1;
> {noformat}
> - This should use a scratch directory inside the table-aes1 location.
> {noformat}
> FROM table-unencrypted
> INSERT OVERWRITE TABLE table-aes128 SELECT id, name
> INSERT OVERWRITE TABLE table-aes256 SELECT id, name
> {noformat}
> - This should use a scratch directory on each of the tables locations.
> - The first SELECT will have its scratch directory on table-aes128 directory.
> - The second SELECT will have its scratch directory on table-aes256 directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7792) Enable tests on Spark branch (2) [Sparch Branch]

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-7792:
---
Component/s: Spark

> Enable tests on Spark branch (2) [Sparch Branch]
> 
>
> Key: HIVE-7792
> URL: https://issues.apache.org/jira/browse/HIVE-7792
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Brock Noland
>Assignee: Venki Korukanti
> Fix For: 1.1.0
>
> Attachments: HIVE-7792.1-spark.patch
>
>
> This jira is to enable *most* of the tests below. If tests don't pass because 
> of some unsupported feature, ensure that a JIRA exists and move on.
> {noformat}
> limit_pushdown.q,\
>   load_dyn_part1.q,\
>   load_dyn_part2.q,\
>   load_dyn_part3.q,\
>   mapjoin_mapjoin.q,\
>   mapreduce1.q,\
>   mapreduce2.q,\
>   merge1.q,\
>   merge2.q,\
>   metadata_only_queries.q,\
>   optimize_nullscan.q,\
>   orc_analyze.q,\
>   orc_merge1.q,\
>   orc_merge2.q,\
>   orc_merge3.q,\
>   orc_merge4.q,\
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9425) Add jar/file doesn't work with yarn-cluster mode [Spark Branch]

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-9425:
---
Component/s: Spark

> Add jar/file doesn't work with yarn-cluster mode [Spark Branch]
> ---
>
> Key: HIVE-9425
> URL: https://issues.apache.org/jira/browse/HIVE-9425
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark, spark-branch
>Reporter: Xiaomin Zhang
>Assignee: Rui Li
> Fix For: 1.1.0
>
> Attachments: HIVE-9425.1-spark.patch
>
>
> {noformat}
> 15/01/20 00:27:31 INFO cluster.YarnClusterScheduler: 
> YarnClusterScheduler.postStartHook done
> 15/01/20 00:27:31 ERROR spark.SparkContext: Error adding jar 
> (java.io.FileNotFoundException: hive-exec-0.15.0-SNAPSHOT.jar (No such file 
> or directory)), was the --addJars option used?
> 15/01/20 00:27:31 ERROR spark.SparkContext: Error adding jar 
> (java.io.FileNotFoundException: opennlp-maxent-3.0.3.jar (No such file or 
> directory)), was the --addJars option used?
> 15/01/20 00:27:31 ERROR spark.SparkContext: Error adding jar 
> (java.io.FileNotFoundException: bigbenchqueriesmr.jar (No such file or 
> directory)), was the --addJars option used?
> 15/01/20 00:27:31 ERROR spark.SparkContext: Error adding jar 
> (java.io.FileNotFoundException: opennlp-tools-1.5.3.jar (No such file or 
> directory)), was the --addJars option used?
> 15/01/20 00:27:31 ERROR spark.SparkContext: Error adding jar 
> (java.io.FileNotFoundException: jcl-over-slf4j-1.7.5.jar (No such file or 
> directory)), was the --addJars option used?
> 15/01/20 00:27:31 INFO client.RemoteDriver: Received job request 
> fef081b0-5408-4804-9531-d131fdd628e6
> 15/01/20 00:27:31 INFO Configuration.deprecation: mapred.max.split.size is 
> deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
> 15/01/20 00:27:31 INFO Configuration.deprecation: mapred.min.split.size is 
> deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
> 15/01/20 00:27:31 INFO client.RemoteDriver: Failed to run job 
> fef081b0-5408-4804-9531-d131fdd628e6
> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find 
> class: de.bankmark.bigbench.queries.q10.SentimentUDF
> Serialization trace:
> genericUDTF (org.apache.hadoop.hive.ql.plan.UDTFDesc)
> conf (org.apache.hadoop.hive.ql.exec.UDTFOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> invertedWorkGraph (org.apache.hadoop.hive.ql.plan.SparkWork)
>   at 
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)
>   at 
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
> {noformat}
> It seems the additional Jar files are not uploaded to DistributedCache, so 
> that the Driver cannot access it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7292) Hive on Spark

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989616#comment-14989616
 ] 

Lars Francke commented on HIVE-7292:


Can we mark this as resolved? It has been merged for 1.1.0 and 1.2.0. There 
will obviously be bugfixes and so on but it seems like the feature itself has 
been delivered and the remaining work can be done in separate JIRAs (as is done 
already)

> Hive on Spark
> -
>
> Key: HIVE-7292
> URL: https://issues.apache.org/jira/browse/HIVE-7292
> Project: Hive
>  Issue Type: Improvement
>  Components: Spark
>Reporter: Xuefu Zhang
>Assignee: Xuefu Zhang
>  Labels: Spark-M1, Spark-M2, Spark-M3, Spark-M4, Spark-M5
> Attachments: Hive-on-Spark.pdf
>
>
> Spark as an open-source data analytics cluster computing framework has gained 
> significant momentum recently. Many Hive users already have Spark installed 
> as their computing backbone. To take advantages of Hive, they still need to 
> have either MapReduce or Tez on their cluster. This initiative will provide 
> user a new alternative so that those user can consolidate their backend. 
> Secondly, providing such an alternative further increases Hive's adoption as 
> it exposes Spark users  to a viable, feature-rich de facto standard SQL tools 
> on Hadoop.
> Finally, allowing Hive to run on Spark also has performance benefits. Hive 
> queries, especially those involving multiple reducer stages, will run faster, 
> thus improving user experience as Tez does.
> This is an umbrella JIRA which will cover many coming subtask. Design doc 
> will be attached here shortly, and will be on the wiki as well. Feedback from 
> the community is greatly appreciated!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-11941) Update committer list

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-11941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907766#comment-14907766
 ] 

Lars Francke commented on HIVE-11941:
-

Thanks! Sure:

* Name: Lars Francke
* Username: larsfrancke
* Organization: Freelancer / http://lars-francke.de/en/

> Update committer list
> -
>
> Key: HIVE-11941
> URL: https://issues.apache.org/jira/browse/HIVE-11941
> Project: Hive
>  Issue Type: Bug
>Reporter: Chaoyu Tang
>Priority: Minor
> Attachments: HIVE-11941.patch
>
>
> Please update the committer list in http://hive.apache.org/people.html:
> ---
> Name: Chaoyu Tang
> Apache ID: ctang
> Organization: Cloudera (www.cloudera.com)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8342) Potential null dereference in ColumnTruncateMapper#jobClose()

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791307#comment-14791307
 ] 

Lars Francke commented on HIVE-8342:


Hey [~tedyu] I get notifications about this issue every once in a while because 
you seemingly change something but it looks like you're not actually changing 
anything. Is this a JIRA problem?

> Potential null dereference in ColumnTruncateMapper#jobClose()
> -
>
> Key: HIVE-8342
> URL: https://issues.apache.org/jira/browse/HIVE-8342
> Project: Hive
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: skrho
>Priority: Minor
> Attachments: HIVE-8342_001.patch, HIVE-8342_002.patch
>
>
> {code}
> Utilities.mvFileToFinalPath(outputPath, job, success, LOG, dynPartCtx, 
> null,
>   reporter);
> {code}
> Utilities.mvFileToFinalPath() calls createEmptyBuckets() where conf is 
> dereferenced:
> {code}
> boolean isCompressed = conf.getCompressed();
> TableDesc tableInfo = conf.getTableInfo();
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726968#comment-14726968
 ] 

Lars Francke commented on HIVE-11708:
-

That's true. Good example, thank you.

I'll try to take a look at this and will assign to me but if anyone gets to it 
before me please don't hesitate to reassign.

> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>
> According to Language Manual UDF, logical operators returns NULL if one of 
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
>  cannot be cast to 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException: 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
>  cannot be cast to 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
>   at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
>   at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
>   at 
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
>   at 
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
>   at 
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
>   at 
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
>   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
> {noformat}
> I confirmed that Hive 1.2.1 of HDP2.3 Sandbox also raises this exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726859#comment-14726859
 ] 

Lars Francke commented on HIVE-11708:
-

I tested this and it looks like it only happens when you use the literal 
"NULL". When you AND two columns together one of which happens to have NULL 
values it seems to work as intended.

Are you seeing the same?

I agree that it's still a bug but if this is the only scenario where it happens 
it'd be an edge case. Or are you seeing this for other queries as well?

> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>
> According to Language Manual UDF, logical operators returns NULL if one of 
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
>  cannot be cast to 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException: 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
>  cannot be cast to 
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
>   at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
>   at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
>   at 
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
>   at 
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
>   at 
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
>   at 
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
>   at 
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
>   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
> {noformat}
> I confirmed that Hive 1.2.1 of HDP2.3 Sandbox also raises this exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-11653) Beeline asks for password even when connecting with Kerberos

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14714510#comment-14714510
 ] 

Lars Francke commented on HIVE-11653:
-

I think I'd like to take a stab at this if it's alright with you? I've been 
annoyed by this ever since beeline was introduced :)

We need to make a decision on what we want to do though.

We could just ignore it if a user doesn't provide username and password. If 
Hive is configured to accept it it'll fail with an authentication error anyway. 
On the other hand it provides a way to not pass in the password in a way that 
ends up in the history or on screen.

Any ideas on how to best handle this?

 Beeline asks for password even when connecting with Kerberos
 

 Key: HIVE-11653
 URL: https://issues.apache.org/jira/browse/HIVE-11653
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.14.0
 Environment: Kerberos and Hive
Reporter: Loïc C. Chanel

 When connecting to HiveServer via Beeline, Beeline asks for a password even 
 if Kerberos is enabled and there is a ticket in cache (kinit have been 
 successfully executed, as klist shows the ticket is in cache).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8007) Clean up Thrift definitions

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710182#comment-14710182
 ] 

Lars Francke commented on HIVE-8007:


Oh...no... I did not regenerate the files as I could not get Thrift to build :(

At least I was correct about the build failure being unrelated ;-)

I'll try again to get Thrift working, thanks for the heads up!

 Clean up Thrift definitions
 ---

 Key: HIVE-8007
 URL: https://issues.apache.org/jira/browse/HIVE-8007
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8007.1.patch, HIVE-8007.2.patch, HIVE-8007.3.patch


 This patch changes the following:
 * Currently the thrift file uses {{//}} to denote comments. Thrift 
 understands the {{/** ... */}} syntax and converts that into documentation in 
 the generated code. This patch changes the syntax
 * Change tabs to spaces
 * Consistent indentation
 * Minor whitespace and/or formatting issues
 There should be no changes to functionality at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8007) Clean up Thrift definitions

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8007:
---
Attachment: HIVE-8007.3.patch

 Clean up Thrift definitions
 ---

 Key: HIVE-8007
 URL: https://issues.apache.org/jira/browse/HIVE-8007
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8007.1.patch, HIVE-8007.2.patch, HIVE-8007.3.patch


 This patch changes the following:
 * Currently the thrift file uses {{//}} to denote comments. Thrift 
 understands the {{/** ... */}} syntax and converts that into documentation in 
 the generated code. This patch changes the syntax
 * Change tabs to spaces
 * Consistent indentation
 * Minor whitespace and/or formatting issues
 There should be no changes to functionality at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700926#comment-14700926
 ] 

Lars Francke commented on HIVE-8583:


Rebased patch

 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch, HIVE-8583.3.patch, 
 HIVE-8583.4.patch, HIVE-8583.5.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this
 * Tabs to Spaces conversion



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8583:
---
Attachment: HIVE-8583.5.patch

 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch, HIVE-8583.3.patch, 
 HIVE-8583.4.patch, HIVE-8583.5.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this
 * Tabs to Spaces conversion



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-3619) Hive JDBC driver should return a proper update-count of rows affected by query

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700921#comment-14700921
 ] 

Lars Francke commented on HIVE-3619:


This has been changed in the meantime to always return -1 but it'd be great if 
Hive could return the correct number when it knows it.

 Hive JDBC driver should return a proper update-count of rows affected by query
 --

 Key: HIVE-3619
 URL: https://issues.apache.org/jira/browse/HIVE-3619
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.9.0
Reporter: Harsh J
Priority: Minor
 Attachments: HIVE-3619.patch


 HiveStatement.java currently has an explicit 0 return:
 public int getUpdateCount() throws SQLException { return 0; }
 Ideally we ought to emit the exact number of rows affected by the query 
 statement itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8007) Clean up Thrift definitions

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702109#comment-14702109
 ] 

Lars Francke commented on HIVE-8007:


Test failure is unrelated

 Clean up Thrift definitions
 ---

 Key: HIVE-8007
 URL: https://issues.apache.org/jira/browse/HIVE-8007
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8007.1.patch, HIVE-8007.2.patch, HIVE-8007.3.patch


 This patch changes the following:
 * Currently the thrift file uses {{//}} to denote comments. Thrift 
 understands the {{/** ... */}} syntax and converts that into documentation in 
 the generated code. This patch changes the syntax
 * Change tabs to spaces
 * Consistent indentation
 * Minor whitespace and/or formatting issues
 There should be no changes to functionality at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8007) Clean up Thrift definitions

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8007:
---
Summary: Clean up Thrift definitions  (was: Use proper Thrift comments)

 Clean up Thrift definitions
 ---

 Key: HIVE-8007
 URL: https://issues.apache.org/jira/browse/HIVE-8007
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8007.1.patch, HIVE-8007.2.patch


 Currently the thrift file uses {{//}} to denote comments. Thrift understands 
 the {{/** ... */}} syntax and converts that into documentation in the 
 generated code. This patch changes the syntax.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8007) Clean up Thrift definitions

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8007:
---
Description: 
This patch changes the following:

* Currently the thrift file uses {{//}} to denote comments. Thrift understands 
the {{/** ... */}} syntax and converts that into documentation in the generated 
code. This patch changes the syntax
* Change tabs to spaces
* Consistent indentation
* Minor whitespace and/or formatting issues

There should be no changes to functionality at all.

  was:Currently the thrift file uses {{//}} to denote comments. Thrift 
understands the {{/** ... */}} syntax and converts that into documentation in 
the generated code. This patch changes the syntax.


 Clean up Thrift definitions
 ---

 Key: HIVE-8007
 URL: https://issues.apache.org/jira/browse/HIVE-8007
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8007.1.patch, HIVE-8007.2.patch


 This patch changes the following:
 * Currently the thrift file uses {{//}} to denote comments. Thrift 
 understands the {{/** ... */}} syntax and converts that into documentation in 
 the generated code. This patch changes the syntax
 * Change tabs to spaces
 * Consistent indentation
 * Minor whitespace and/or formatting issues
 There should be no changes to functionality at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8007) Clean up Thrift definitions

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555813#comment-14555813
 ] 

Lars Francke commented on HIVE-8007:


No idea how to get it to run again but I'm certain those test failures are 
unrelated because this is a whitespace only change.

 Clean up Thrift definitions
 ---

 Key: HIVE-8007
 URL: https://issues.apache.org/jira/browse/HIVE-8007
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8007.1.patch, HIVE-8007.2.patch


 This patch changes the following:
 * Currently the thrift file uses {{//}} to denote comments. Thrift 
 understands the {{/** ... */}} syntax and converts that into documentation in 
 the generated code. This patch changes the syntax
 * Change tabs to spaces
 * Consistent indentation
 * Minor whitespace and/or formatting issues
 There should be no changes to functionality at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7938) Fail early with good error message in LazyBinaryString#init

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-7938:
---
Attachment: HIVE-7938.2.patch

Rebased to master

 Fail early with good error message in LazyBinaryString#init
 ---

 Key: HIVE-7938
 URL: https://issues.apache.org/jira/browse/HIVE-7938
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-7938.1.patch, HIVE-7938.2.patch


 This method seems to be called with illegal parameters from time to time. 
 HIVE-3945 is one example and google shows more. This patch adds a better 
 exception.
 I thought about also adding a log statement but I'm not too sure about that. 
 On the other hand the exception message might be swallowed...open for 
 suggestions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke resolved HIVE-8222.

Resolution: Won't Fix

 CBO Trunk Merge: Fix Check Style issues
 ---

 Key: HIVE-8222
 URL: https://issues.apache.org/jira/browse/HIVE-8222
 Project: Hive
  Issue Type: Sub-task
  Components: CBO
Reporter: Laljo John Pullokkaran
Assignee: Lars Francke
 Attachments: HIVE-8222.1.patch, HIVE-8222.2.patch, HIVE-8222.3.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8007) Use proper Thrift comments

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8007:
---
Attachment: HIVE-8007.2.patch

 Use proper Thrift comments
 --

 Key: HIVE-8007
 URL: https://issues.apache.org/jira/browse/HIVE-8007
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8007.1.patch, HIVE-8007.2.patch


 Currently the thrift file uses {{//}} to denote comments. Thrift understands 
 the {{/** ... */}} syntax and converts that into documentation in the 
 generated code. This patch changes the syntax.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8583:
---
Attachment: HIVE-8583.3.patch

Next try, let's see if this works :) . The additional whitespace changes are 
conversions from tabs to space. I'm happy to rip them out and submit as 
separate JIRA if you think that's better.

 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch, HIVE-8583.3.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Minor style modification (missorted modifiers)
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Removes replaceAll
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8583:
---
Description: 
[~alangates] added the following in HIVE-8341:

{code}
String bl = 
hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
if (bl != null  bl.length()  0) {
  String[] bls = bl.split(,);
  for (String b : bls) {
b.replaceAll(., _);
blackListedConfEntries.add(b);
  }
}
{code}

The {{replaceAll}} call is confusing as its result is not used at all.

This patch contains the following:

* Minor style modification (missorted modifiers)
* Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
* Removes replaceAll
* Lets blackListed take a Configuration job as parameter which allowed me to 
add a test for this
* Tabs to Spaces conversion

  was:
[~alangates] added the following in HIVE-8341:

{code}
String bl = 
hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
if (bl != null  bl.length()  0) {
  String[] bls = bl.split(,);
  for (String b : bls) {
b.replaceAll(., _);
blackListedConfEntries.add(b);
  }
}
{code}

The {{replaceAll}} call is confusing as its result is not used at all.

This patch contains the following:

* Minor style modification (missorted modifiers)
* Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
* Removes replaceAll
* Lets blackListed take a Configuration job as parameter which allowed me to 
add a test for this


 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch, HIVE-8583.3.patch, 
 HIVE-8583.4.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Minor style modification (missorted modifiers)
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Removes replaceAll
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this
 * Tabs to Spaces conversion



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8583:
---
Attachment: HIVE-8583.4.patch

 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch, HIVE-8583.3.patch, 
 HIVE-8583.4.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Minor style modification (missorted modifiers)
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Removes replaceAll
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8583:
---
Description: 
[~alangates] added the following in HIVE-8341:

{code}
String bl = 
hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
if (bl != null  bl.length()  0) {
  String[] bls = bl.split(,);
  for (String b : bls) {
b.replaceAll(., _);
blackListedConfEntries.add(b);
  }
}
{code}

The {{replaceAll}} call is confusing as its result is not used at all.

This patch contains the following:

* Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
* Lets blackListed take a Configuration job as parameter which allowed me to 
add a test for this
* Tabs to Spaces conversion

  was:
[~alangates] added the following in HIVE-8341:

{code}
String bl = 
hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
if (bl != null  bl.length()  0) {
  String[] bls = bl.split(,);
  for (String b : bls) {
b.replaceAll(., _);
blackListedConfEntries.add(b);
  }
}
{code}

The {{replaceAll}} call is confusing as its result is not used at all.

This patch contains the following:

* Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
* Removes replaceAll
* Lets blackListed take a Configuration job as parameter which allowed me to 
add a test for this
* Tabs to Spaces conversion


 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch, HIVE-8583.3.patch, 
 HIVE-8583.4.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this
 * Tabs to Spaces conversion



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543334#comment-14543334
 ] 

Lars Francke commented on HIVE-8583:


Well, apparently you switched to Git in the meantime (not documented in Wiki  
Homepage). I'll need ro rebase and update patch.

 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Minor style modification (missorted modifiers)
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Removes replaceAll
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8583:
---
Description: 
[~alangates] added the following in HIVE-8341:

{code}
String bl = 
hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
if (bl != null  bl.length()  0) {
  String[] bls = bl.split(,);
  for (String b : bls) {
b.replaceAll(., _);
blackListedConfEntries.add(b);
  }
}
{code}

The {{replaceAll}} call is confusing as its result is not used at all.

This patch contains the following:

* Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
* Removes replaceAll
* Lets blackListed take a Configuration job as parameter which allowed me to 
add a test for this
* Tabs to Spaces conversion

  was:
[~alangates] added the following in HIVE-8341:

{code}
String bl = 
hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
if (bl != null  bl.length()  0) {
  String[] bls = bl.split(,);
  for (String b : bls) {
b.replaceAll(., _);
blackListedConfEntries.add(b);
  }
}
{code}

The {{replaceAll}} call is confusing as its result is not used at all.

This patch contains the following:

* Minor style modification (missorted modifiers)
* Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
* Removes replaceAll
* Lets blackListed take a Configuration job as parameter which allowed me to 
add a test for this
* Tabs to Spaces conversion


 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch, HIVE-8583.3.patch, 
 HIVE-8583.4.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Removes replaceAll
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this
 * Tabs to Spaces conversion



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542973#comment-14542973
 ] 

Lars Francke commented on HIVE-8583:


I dropped the ball on this one but did rebase and uploaded a new patch.

 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Minor style modification (missorted modifiers)
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Removes replaceAll
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8583) HIVE-8341 Cleanup Test for hive.script.operator.env.blacklist

[Lars Francke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-8583:
---
Attachment: HIVE-8583.2.patch

 HIVE-8341 Cleanup  Test for hive.script.operator.env.blacklist
 ---

 Key: HIVE-8583
 URL: https://issues.apache.org/jira/browse/HIVE-8583
 Project: Hive
  Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-8583.1.patch, HIVE-8583.2.patch


 [~alangates] added the following in HIVE-8341:
 {code}
 String bl = 
 hconf.get(HiveConf.ConfVars.HIVESCRIPT_ENV_BLACKLIST.toString());
 if (bl != null  bl.length()  0) {
   String[] bls = bl.split(,);
   for (String b : bls) {
 b.replaceAll(., _);
 blackListedConfEntries.add(b);
   }
 }
 {code}
 The {{replaceAll}} call is confusing as its result is not used at all.
 This patch contains the following:
 * Minor style modification (missorted modifiers)
 * Adds reading of default value for HIVESCRIPT_ENV_BLACKLIST
 * Removes replaceAll
 * Lets blackListed take a Configuration job as parameter which allowed me to 
 add a test for this



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9877) Beeline cannot run multiple statements in the same row

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502127#comment-14502127
 ] 

Lars Francke commented on HIVE-9877:


Thanks [~ctang.ma]. I see your points and while I disagree with the conclusion 
I won't argue the change. I just wanted to raise it in case you weren't aware 
of the previous discussions. Let's keep it the way it is now.

 Beeline cannot run multiple statements in the same row
 --

 Key: HIVE-9877
 URL: https://issues.apache.org/jira/browse/HIVE-9877
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.12.0
 Environment: Oracle Linux 6.5, x64, Cloudera 5.1.3, Hive 0.12.0
Reporter: Zoltan Fedor
Assignee: Chaoyu Tang
  Labels: TODOC1.2
 Fix For: 1.2.0

 Attachments: HIVE-9877.patch, HIVE-9877.patch


 I'm trying to switch from hive cli to beeline and found the below working 
 with hive cli, but not with beeline.
 This works in hive cli:
 $ hive -e USE my_db;SHOW TABLES; 
 The same does not work in beeline:
 $ beeline -u jdbc:hive2://my_server.com -n my_user -p my_password -e USE 
 my_db;SHOW TABLES;
 Error: Error while compiling statement: FAILED: ParseException line 1:9 
 missing EOF at ';' near 'my_db' (state=42000,code=4)
 Beeline version 0.12.0-cdh5.1.3 by Apache Hive 
 I have also tried with beeline -f [filename]
 The issue is the same, except (!) when the two statements are listed in 
 separate lines in the file supplied via the -f parameter.
 So when using 
 beeline -f my.hql
 This works:
 my.hql:
 USE my_db;
 SHOW TABLES;
 This does not work:
 my.hql:
 USE my_db;SHOW TABLES;
 $ beeline -u jdbc:hive2://my_server.com -n my_user -p my_password -f my.hql
 Connected to: Apache Hive (version 0.12.0-cdh5.1.3)
 Driver: Hive JDBC (version 0.12.0-cdh5.1.3)
 Transaction isolation: TRANSACTION_REPEATABLE_READ
 Beeline version 0.12.0-cdh5.1.3 by Apache Hive
 0: jdbc:hive2://my_server.com USE my_db;SHOW TABLES;
 Error: Error while compiling statement: FAILED: ParseException line 1:9 
 missing EOF at ';' near 'my_db' (state=42000,code=4)
 Closing: org.apache.hive.jdbc.HiveConnection
 How to reproduce:
 Run any type of multiple statements with beeline where the statements are in 
 the same line separated by ; whether using beeline -e [statement] or 
 beeline -f [file]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9877) Beeline cannot run multiple statements in the same row

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369267#comment-14369267
 ] 

Lars Francke commented on HIVE-9877:


I only took a cursory look at the patch but isn't this the same as HIVE-7348 
which was closed without implementing the change?

Just splitting on semicolon is not a good option at least for -e (and 
probably for -f as well). If this is what this patch does and the arguments 
from the other issue are still valid I'd be in favor of reverting this patch 
for now.

 Beeline cannot run multiple statements in the same row
 --

 Key: HIVE-9877
 URL: https://issues.apache.org/jira/browse/HIVE-9877
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.12.0
 Environment: Oracle Linux 6.5, x64, Cloudera 5.1.3, Hive 0.12.0
Reporter: Zoltan Fedor
Assignee: Chaoyu Tang
  Labels: TODOC1.2
 Fix For: 1.2.0

 Attachments: HIVE-9877.patch, HIVE-9877.patch


 I'm trying to switch from hive cli to beeline and found the below working 
 with hive cli, but not with beeline.
 This works in hive cli:
 $ hive -e USE my_db;SHOW TABLES; 
 The same does not work in beeline:
 $ beeline -u jdbc:hive2://my_server.com -n my_user -p my_password -e USE 
 my_db;SHOW TABLES;
 Error: Error while compiling statement: FAILED: ParseException line 1:9 
 missing EOF at ';' near 'my_db' (state=42000,code=4)
 Beeline version 0.12.0-cdh5.1.3 by Apache Hive 
 I have also tried with beeline -f [filename]
 The issue is the same, except (!) when the two statements are listed in 
 separate lines in the file supplied via the -f parameter.
 So when using 
 beeline -f my.hql
 This works:
 my.hql:
 USE my_db;
 SHOW TABLES;
 This does not work:
 my.hql:
 USE my_db;SHOW TABLES;
 $ beeline -u jdbc:hive2://my_server.com -n my_user -p my_password -f my.hql
 Connected to: Apache Hive (version 0.12.0-cdh5.1.3)
 Driver: Hive JDBC (version 0.12.0-cdh5.1.3)
 Transaction isolation: TRANSACTION_REPEATABLE_READ
 Beeline version 0.12.0-cdh5.1.3 by Apache Hive
 0: jdbc:hive2://my_server.com USE my_db;SHOW TABLES;
 Error: Error while compiling statement: FAILED: ParseException line 1:9 
 missing EOF at ';' near 'my_db' (state=42000,code=4)
 Closing: org.apache.hive.jdbc.HiveConnection
 How to reproduce:
 Run any type of multiple statements with beeline where the statements are in 
 the same line separated by ; whether using beeline -e [statement] or 
 beeline -f [file]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)