[jira] [Updated] (HIVE-2897) Remove filter operator of which all the predicates have pushed down to storage handler

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2897:
-

Status: Open  (was: Patch Available)

> Remove filter operator of which all the predicates have pushed down to 
> storage handler
> --
>
> Key: HIVE-2897
> URL: https://issues.apache.org/jira/browse/HIVE-2897
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-2897.D2427.1.patch, HIVE-2897.D2427.2.patch
>
>
> If all the predicates have pushed down to StorageHandler in TS, it should be 
> removed (or make empty). Might it be intentional? 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3040) Hive Exec jar should not include Thrift

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280609#comment-13280609
 ] 

Ashutosh Chauhan commented on HIVE-3040:


This should be done in conjunction with HIVE-3017

> Hive Exec jar should not include Thrift
> ---
>
> Key: HIVE-3040
> URL: https://issues.apache.org/jira/browse/HIVE-3040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.8.1
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
> Fix For: 0.8.1
>
> Attachments: HIVE-3040.1.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> hive-exec jar includes Thrift classes even though it does not need them. This 
> can create problems because it can load some wrong version of Thrift and 
> other  jars that need Thrift get stuck with the wrong version. We will remove 
> Thrift from this jar.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3040) Hive Exec jar should not include Thrift

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280602#comment-13280602
 ] 

Carl Steinbach commented on HIVE-3040:
--

hive-exec also includes all of the serde classes, some of which depend on 
Thrift (e.g. TCTLSeparatedProtocol, DynamicSerDe, etc). If you remove Thrift 
from hive-exec then you also need to remove these classes and the associated 
test cases from hive-serde.

> Hive Exec jar should not include Thrift
> ---
>
> Key: HIVE-3040
> URL: https://issues.apache.org/jira/browse/HIVE-3040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.8.1
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
> Fix For: 0.8.1
>
> Attachments: HIVE-3040.1.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> hive-exec jar includes Thrift classes even though it does not need them. This 
> can create problems because it can load some wrong version of Thrift and 
> other  jars that need Thrift get stuck with the wrong version. We will remove 
> Thrift from this jar.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-3039) Move Hive admin documentation from wiki to version control

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan reassigned HIVE-3039:
--

Assignee: Lefty Leverenz

For HiveConfiguration instead of listing all the configuration, its better to 
auto-generate them from hive-default.xml.template

> Move Hive admin documentation from wiki to version control
> --
>
> Key: HIVE-3039
> URL: https://issues.apache.org/jira/browse/HIVE-3039
> Project: Hive
>  Issue Type: Task
>  Components: Documentation
>Reporter: Lefty Leverenz
>Assignee: Lefty Leverenz
>  Labels: documentation
> Attachments: HIVE-3039.1.patch
>
>
> Move the Hive administrator documentation from the wiki to version control.  
> The wiki doc set for administrators currently has nine html files that need 
> to be converted to xml:
> Installing Hive
> Configuring Hive
> Setting up Metastore
> Setting up Hive Web Interface
> Setting up Thrift Hive Server
> Setting up Hive JDBC Server
> Setting up Hive ODBC Server
> Hive on Amazon Web Services
> Hive on Amazon ElasticMapReduce

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3040) Hive Exec jar should not include Thrift

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-3040:


Attachment: HIVE-3040.1.patch

> Hive Exec jar should not include Thrift
> ---
>
> Key: HIVE-3040
> URL: https://issues.apache.org/jira/browse/HIVE-3040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.8.1
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
> Fix For: 0.8.1
>
> Attachments: HIVE-3040.1.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> hive-exec jar includes Thrift classes even though it does not need them. This 
> can create problems because it can load some wrong version of Thrift and 
> other  jars that need Thrift get stuck with the wrong version. We will remove 
> Thrift from this jar.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3042) Thrift classes do not need to be passed to the mappers and reducers

[Paul Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280584#comment-13280584
 ] 

Paul Yang commented on HIVE-3042:
-

Duplicate of HIVE-3040

> Thrift classes do not need to be passed to the mappers and reducers
> ---
>
> Key: HIVE-3042
> URL: https://issues.apache.org/jira/browse/HIVE-3042
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.10.0
>Reporter: Namit Jain
>Assignee: Paul Yang
> Fix For: 0.10.0
>
> Attachments: HIVE-3042.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3042) Thrift classes do not need to be passed to the mappers and reducers

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-3042:


Resolution: Duplicate
Status: Resolved  (was: Patch Available)

> Thrift classes do not need to be passed to the mappers and reducers
> ---
>
> Key: HIVE-3042
> URL: https://issues.apache.org/jira/browse/HIVE-3042
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.10.0
>Reporter: Namit Jain
>Assignee: Paul Yang
> Fix For: 0.10.0
>
> Attachments: HIVE-3042.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HIVE-2989) Adding Table Links to Hive

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280575#comment-13280575
 ] 

Namit Jain edited comment on HIVE-2989 at 5/21/12 10:42 PM:


The goal is to do much better accounting. Please take a look at
https://cwiki.apache.org/confluence/display/Hive/Links


  was (Author: namit):
The goal is to do much better accounting. Take a look at
https://cwiki.apache.org/confluence/display/Hive/Links

  
> Adding Table Links to Hive
> --
>
> Key: HIVE-2989
> URL: https://issues.apache.org/jira/browse/HIVE-2989
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore, Query Processor, Security
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
> Fix For: 0.10.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> This will add Table Links to Hive. This will be an alternate mechanism for a 
> user to access tables and data in a database that is different from the one 
> he is associated with. This feature can be used to provide access control (if 
> access to databasename.tablename in queries and "use database X" is turned 
> off in conjunction).
> If db X wants to access one or more partitions from table T in db Y, the user 
> will issue:
> CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
> New partitions added to T will automatically be added to the link as well and 
> become available to X. However, if the link is specified to be static, that 
> will not be the case. The X user will then have to explicitly import each 
> partition of T that he needs. The command above will not actually make any 
> existing partitions of T available to X. Instead, we provide the following 
> command to add an existing partition to a link:
> ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
> The user will need to execute the above for each existing partition that 
> needs to be imported. For future partitions, Hive will take care of this. An 
> imported partition can be dropped from a link using a similar command. We 
> just specify "DROP" instead of "ADD". For querying the linked table, the X 
> user will refer to it as T@Y. Link Tables will only have read access and not 
> be writable. The entire Table Link alongwith all its imported partitions can 
> be dropped as follows:
> DROP LINK TO T@Y
> The above commands are purely MetaStore operations. The implementation will 
> rely on replicating the entire partition metadata when a partition is added 
> to a link.  For every link that is created, we will add a new row to table 
> TBLS. The TBL_TYPE column will have a new kind of value "LINK_TABLE" (or 
> "STATIC_LINK_TABLE" if the link has been specified as static). A new column 
> LINK_TBL_ID will be added which will contain the id of the imported table. It 
> will be NULL for all other table types including the regular managed tables. 
> When a partition is added to a link, the new row in the table PARTITIONS will 
> point to the LINK_TABLE in the same database  and not the master table in the 
> other database. We will replicate all the metadata for this partition from 
> the master database. The advantage of this approach is that fewer changes 
> will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
> like "SHOW TABLES" and "SHOW PARTITIONS" will work as expected for 
> LINK_TABLEs too. Of course, even though the metadata is not shared, the 
> underlying data on disk is still shared. Hive still needs to know that when 
> dropping a partition which belongs to a LINK_TABLE, it should not drop the 
> underlying data from HDFS. Views and external tables cannot be imported from 
> one database to another.
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280575#comment-13280575
 ] 

Namit Jain commented on HIVE-2989:
--

The goal is to do much better accounting. Take a look at
https://cwiki.apache.org/confluence/display/Hive/Links


> Adding Table Links to Hive
> --
>
> Key: HIVE-2989
> URL: https://issues.apache.org/jira/browse/HIVE-2989
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore, Query Processor, Security
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
> Fix For: 0.10.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> This will add Table Links to Hive. This will be an alternate mechanism for a 
> user to access tables and data in a database that is different from the one 
> he is associated with. This feature can be used to provide access control (if 
> access to databasename.tablename in queries and "use database X" is turned 
> off in conjunction).
> If db X wants to access one or more partitions from table T in db Y, the user 
> will issue:
> CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
> New partitions added to T will automatically be added to the link as well and 
> become available to X. However, if the link is specified to be static, that 
> will not be the case. The X user will then have to explicitly import each 
> partition of T that he needs. The command above will not actually make any 
> existing partitions of T available to X. Instead, we provide the following 
> command to add an existing partition to a link:
> ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
> The user will need to execute the above for each existing partition that 
> needs to be imported. For future partitions, Hive will take care of this. An 
> imported partition can be dropped from a link using a similar command. We 
> just specify "DROP" instead of "ADD". For querying the linked table, the X 
> user will refer to it as T@Y. Link Tables will only have read access and not 
> be writable. The entire Table Link alongwith all its imported partitions can 
> be dropped as follows:
> DROP LINK TO T@Y
> The above commands are purely MetaStore operations. The implementation will 
> rely on replicating the entire partition metadata when a partition is added 
> to a link.  For every link that is created, we will add a new row to table 
> TBLS. The TBL_TYPE column will have a new kind of value "LINK_TABLE" (or 
> "STATIC_LINK_TABLE" if the link has been specified as static). A new column 
> LINK_TBL_ID will be added which will contain the id of the imported table. It 
> will be NULL for all other table types including the regular managed tables. 
> When a partition is added to a link, the new row in the table PARTITIONS will 
> point to the LINK_TABLE in the same database  and not the master table in the 
> other database. We will replicate all the metadata for this partition from 
> the master database. The advantage of this approach is that fewer changes 
> will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
> like "SHOW TABLES" and "SHOW PARTITIONS" will work as expected for 
> LINK_TABLEs too. Of course, even though the metadata is not shared, the 
> underlying data on disk is still shared. Hive still needs to know that when 
> dropping a partition which belongs to a LINK_TABLE, it should not drop the 
> underlying data from HDFS. Views and external tables cannot be imported from 
> one database to another.
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: new feature in hive: links

[Namit Jain]

Yes, regions have been abandoned.
The cross data center writes for the metastases (in case of multiple
regions) 
turns out to a deal-breaker.



On 5/21/12 2:30 PM, "Edward Capriolo"  wrote:

>Can I ask a possibly related question? How does this fit in with hive
>regions? Have regions been abandoned?
>
>On Mon, May 21, 2012 at 3:30 PM, Namit Jain  wrote:
>>
>>
>> There is a requirement for a new feature, for which Sambavi has written
>>a
>> detailed overview
>>https://cwiki.apache.org/confluence/display/Hive/Links.
>>
>> We would like to get some core concepts out of it, and implement them
>>in open source hive, so that the whole community gets it.
>> There are new concepts, so please comment, and we can take it forward
>>accordingly.
>>
>>
>> Thanks,
>> -namit
>>
>>
>>
>>

Re: new feature in hive: links

[Edward Capriolo]

Can I ask a possibly related question? How does this fit in with hive
regions? Have regions been abandoned?

On Mon, May 21, 2012 at 3:30 PM, Namit Jain  wrote:
>
>
> There is a requirement for a new feature, for which Sambavi has written a
> detailed overview  https://cwiki.apache.org/confluence/display/Hive/Links.
>
> We would like to get some core concepts out of it, and implement them in open 
> source hive, so that the whole community gets it.
> There are new concepts, so please comment, and we can take it forward 
> accordingly.
>
>
> Thanks,
> -namit
>
>
>
>

[jira] [Created] (HIVE-3043) escape all characters as mysql in hive

[Namit Jain (JIRA)]

Namit Jain created HIVE-3043:


 Summary: escape all characters as mysql in hive
 Key: HIVE-3043
 URL: https://issues.apache.org/jira/browse/HIVE-3043
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain


escape all the characters across custom transforms as mysql


In HIVE-3030, when hive.transform.escape.input is set to true, hive only 
escapes the following:

tab
carriage return
new line
\


It would be useful to be compatible with mysql:

http://dev.mysql.com/doc/refman/5.0/en/string-literals.html#character-escape-sequences


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3030) escape more chars for script operator

[Siying Dong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280470#comment-13280470
 ] 

Siying Dong commented on HIVE-3030:
---

Tests look good to me. Will run the test suites. Let's open a follow-up JIRA to 
escape a more complete list of characters.

> escape more chars for script operator
> -
>
> Key: HIVE-3030
> URL: https://issues.apache.org/jira/browse/HIVE-3030
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> Only new line was being escaped.
> The same behavior needs to be done for carriage returns, and tabs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280468#comment-13280468
 ] 

Ashutosh Chauhan commented on HIVE-2585:


Bump. Got hit by this today again. Carl, do you have any further suggestions. 
Else, I think its ready to go in?

> Collapse hive.metastore.uris and hive.metastore.local
> -
>
> Key: HIVE-2585
> URL: https://issues.apache.org/jira/browse/HIVE-2585
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-2585.D2559.1.patch, HIVE-2585.D2559.2.patch
>
>
> We should just have hive.metastore.uris. If it is empty, we shall assume 
> local mode, if non-empty we shall use that string to connect to remote 
> metastore. Having two different keys for same information is confusing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3030) escape more chars for script operator

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280460#comment-13280460
 ] 

Namit Jain commented on HIVE-3030:
--

Added more tests https://reviews.facebook.net/D3261


> escape more chars for script operator
> -
>
> Key: HIVE-3030
> URL: https://issues.apache.org/jira/browse/HIVE-3030
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> Only new line was being escaped.
> The same behavior needs to be done for carriage returns, and tabs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

[Jakob Homan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan updated HIVE-895:
-

Attachment: (was: HIVE-895.patch)

> Add SerDe for Avro serialized data
> --
>
> Key: HIVE-895
> URL: https://issues.apache.org/jira/browse/HIVE-895
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Jeff Hammerbacher
>Assignee: Jakob Homan
> Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
> episodes.avro
>
>
> As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
> data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

[Jakob Homan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan updated HIVE-895:
-

Attachment: (was: doctors.avro)

> Add SerDe for Avro serialized data
> --
>
> Key: HIVE-895
> URL: https://issues.apache.org/jira/browse/HIVE-895
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Jeff Hammerbacher
>Assignee: Jakob Homan
> Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
> episodes.avro
>
>
> As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
> data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

[Jakob Homan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan updated HIVE-895:
-

Attachment: (was: episodes.avro)

> Add SerDe for Avro serialized data
> --
>
> Key: HIVE-895
> URL: https://issues.apache.org/jira/browse/HIVE-895
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Jeff Hammerbacher
>Assignee: Jakob Homan
> Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
> episodes.avro
>
>
> As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
> data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3042) Thrift classes do not need to be passed to the mappers and reducers

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-3042:


Fix Version/s: 0.10.0
Affects Version/s: 0.10.0
   Status: Patch Available  (was: Open)

> Thrift classes do not need to be passed to the mappers and reducers
> ---
>
> Key: HIVE-3042
> URL: https://issues.apache.org/jira/browse/HIVE-3042
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.10.0
>Reporter: Namit Jain
>Assignee: Paul Yang
> Fix For: 0.10.0
>
> Attachments: HIVE-3042.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3042) Thrift classes do not need to be passed to the mappers and reducers

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-3042:


Summary: Thrift classes do not need to be passed to the mappers and 
reducers  (was: thrift jars do not need to be passed to the mappers and 
reducers)

> Thrift classes do not need to be passed to the mappers and reducers
> ---
>
> Key: HIVE-3042
> URL: https://issues.apache.org/jira/browse/HIVE-3042
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.10.0
>Reporter: Namit Jain
>Assignee: Paul Yang
> Fix For: 0.10.0
>
> Attachments: HIVE-3042.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3042) thrift jars do not need to be passed to the mappers and reducers

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-3042:


Attachment: HIVE-3042.1.patch

> thrift jars do not need to be passed to the mappers and reducers
> 
>
> Key: HIVE-3042
> URL: https://issues.apache.org/jira/browse/HIVE-3042
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.10.0
>Reporter: Namit Jain
>Assignee: Paul Yang
> Fix For: 0.10.0
>
> Attachments: HIVE-3042.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

[Jakob Homan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan updated HIVE-895:
-

Attachment: HIVE-895.patch
episodes.avro
doctors.avro

Forgot to grant ASF license.  Re-uping.

> Add SerDe for Avro serialized data
> --
>
> Key: HIVE-895
> URL: https://issues.apache.org/jira/browse/HIVE-895
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Jeff Hammerbacher
>Assignee: Jakob Homan
> Attachments: HIVE-895-draft.patch, HIVE-895.patch, HIVE-895.patch, 
> doctors.avro, doctors.avro, episodes.avro, episodes.avro
>
>
> As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
> data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

[Jakob Homan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan updated HIVE-895:
-

Attachment: doctors.avro
episodes.avro
HIVE-895.patch

Final patch.  Swtiching to TBLPROPERTIES rather than SERDEPROPERTIES obviated 
the need for the ql calls previously.  Spent a lot of frustrtating time trying 
to get the phabricator to work (quite surprised that this FB-specific framework 
is kosher in the ASF).  It's up at https://reviews.facebook.net/D3321

> Add SerDe for Avro serialized data
> --
>
> Key: HIVE-895
> URL: https://issues.apache.org/jira/browse/HIVE-895
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Jeff Hammerbacher
>Assignee: Jakob Homan
> Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
> episodes.avro
>
>
> As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
> data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

new feature in hive: links

[Namit Jain]

There is a requirement for a new feature, for which Sambavi has written a
detailed overview  https://cwiki.apache.org/confluence/display/Hive/Links.

We would like to get some core concepts out of it, and implement them in open 
source hive, so that the whole community gets it.
There are new concepts, so please comment, and we can take it forward 
accordingly.


Thanks,
-namit

[jira] [Commented] (HIVE-3030) escape more chars for script operator

[Siying Dong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280399#comment-13280399
 ] 

Siying Dong commented on HIVE-3030:
---

Discussed with Namit offline. He is going to add one more test case now.

> escape more chars for script operator
> -
>
> Key: HIVE-3030
> URL: https://issues.apache.org/jira/browse/HIVE-3030
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> Only new line was being escaped.
> The same behavior needs to be done for carriage returns, and tabs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3030) escape more chars for script operator

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280351#comment-13280351
 ] 

Namit Jain commented on HIVE-3030:
--

https://reviews.facebook.net/D3261 updated - tests added.

@Carl, It seems little clumsy for the user to specify the list of escape 
characters.


> escape more chars for script operator
> -
>
> Key: HIVE-3030
> URL: https://issues.apache.org/jira/browse/HIVE-3030
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> Only new line was being escaped.
> The same behavior needs to be done for carriage returns, and tabs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3042) thrift jars do not need to be passed to the mappers and reducers

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3042:
-

Assignee: Paul Yang  (was: Namit Jain)

> thrift jars do not need to be passed to the mappers and reducers
> 
>
> Key: HIVE-3042
> URL: https://issues.apache.org/jira/browse/HIVE-3042
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Paul Yang
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-3042) thrift jars do not need to be passed to the mappers and reducers

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-3042:


Assignee: Namit Jain

> thrift jars do not need to be passed to the mappers and reducers
> 
>
> Key: HIVE-3042
> URL: https://issues.apache.org/jira/browse/HIVE-3042
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3042) thrift jars do not need to be passed to the mappers and reducers

[Namit Jain (JIRA)]

Namit Jain created HIVE-3042:


 Summary: thrift jars do not need to be passed to the mappers and 
reducers
 Key: HIVE-3042
 URL: https://issues.apache.org/jira/browse/HIVE-3042
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3041) create a timeout for locks

[Namit Jain (JIRA)]

Namit Jain created HIVE-3041:


 Summary: create a timeout for locks
 Key: HIVE-3041
 URL: https://issues.apache.org/jira/browse/HIVE-3041
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain


A common problem that we have seen in hive concurrency is as follows:

A table/partition is being read by a lot of clients.
This leads to starvation of the writer, since new readers are always coming
up. There should be a way to add a configurable timeout for the writer,
default can still be 0 to preserve the current behavior.

This should be very easy to do via the default concurrency manager, 
ZooKeeper, but should be do-able via other implementations (in the future)
also. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2907) Hive error when dropping a table with large number of partitions

[Srinivas (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280317#comment-13280317
 ] 

Srinivas commented on HIVE-2907:


Thanks for the timely clarification.

> Hive error when dropping a table with large number of partitions
> 
>
> Key: HIVE-2907
> URL: https://issues.apache.org/jira/browse/HIVE-2907
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.9.0
> Environment: General. Hive Metastore bug.
>Reporter: Mousom Dhar Gupta
>Assignee: Mousom Dhar Gupta
>Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-2907.1.patch.txt, HIVE-2907.2.patch.txt, 
> HIVE-2907.3.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, 
> HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, 
> HIVE-2907.D2505.6.patch, HIVE-2907.D2505.7.patch
>
>   Original Estimate: 10h
>  Remaining Estimate: 10h
>
> Running into an "Out Of Memory" error when trying to drop a table with 128K 
> partitions.
> The methods dropTable in 
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> encounter out of memory errors 
> when dropping tables with lots of partitions because they try to load the 
> metadata for every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2907) Hive error when dropping a table with large number of partitions

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-2907:
-

Fix Version/s: (was: 0.9.0)
   0.10.0

@Srinivas: This patch was not backported to 0.9.0. The previous fixVersion was 
inaccurate. I have updated the field to make it clear that this patch will 
appear in 0.10.0. Thanks.

> Hive error when dropping a table with large number of partitions
> 
>
> Key: HIVE-2907
> URL: https://issues.apache.org/jira/browse/HIVE-2907
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.9.0
> Environment: General. Hive Metastore bug.
>Reporter: Mousom Dhar Gupta
>Assignee: Mousom Dhar Gupta
>Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-2907.1.patch.txt, HIVE-2907.2.patch.txt, 
> HIVE-2907.3.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, 
> HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, 
> HIVE-2907.D2505.6.patch, HIVE-2907.D2505.7.patch
>
>   Original Estimate: 10h
>  Remaining Estimate: 10h
>
> Running into an "Out Of Memory" error when trying to drop a table with 128K 
> partitions.
> The methods dropTable in 
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> encounter out of memory errors 
> when dropping tables with lots of partitions because they try to load the 
> metadata for every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2907) Hive error when dropping a table with large number of partitions

[Srinivas (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280308#comment-13280308
 ] 

Srinivas commented on HIVE-2907:


Also, the method dropTable in "DDLTask.java"  appears to be missing the fix 
proposed in the JIRA. I am looking at the Hive-0.9.0 source code base. 

> Hive error when dropping a table with large number of partitions
> 
>
> Key: HIVE-2907
> URL: https://issues.apache.org/jira/browse/HIVE-2907
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.9.0
> Environment: General. Hive Metastore bug.
>Reporter: Mousom Dhar Gupta
>Assignee: Mousom Dhar Gupta
>Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2907.1.patch.txt, HIVE-2907.2.patch.txt, 
> HIVE-2907.3.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, 
> HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, 
> HIVE-2907.D2505.6.patch, HIVE-2907.D2505.7.patch
>
>   Original Estimate: 10h
>  Remaining Estimate: 10h
>
> Running into an "Out Of Memory" error when trying to drop a table with 128K 
> partitions.
> The methods dropTable in 
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> encounter out of memory errors 
> when dropping tables with lots of partitions because they try to load the 
> metadata for every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2907) Hive error when dropping a table with large number of partitions

[Srinivas (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280304#comment-13280304
 ] 

Srinivas commented on HIVE-2907:


I downloaded the source code for Hive-0.9.1. However, it appears that 
"ObjectStore.java" is missing the fix that fetches partition-metadata in 
batches. So, it can still cause issues when trying to drop a table with a large 
number of partitions. 

Proposed fix in method dropTable from ObjectStore.java
==
 int partitionBatchSize = HiveConf.getIntVar(getConf(),
 
ConfVars.METASTORE_BATCH_RETRIEVE_MAX);

// call dropPartition on each of the table's partitions to follow the
// procedure for cleanly dropping partitions.
List partsToDelete = listMPartitions(dbName, tableName, 
partitionBatchSize);
while (true){
  if (partsToDelete != null || partsToDelete.isEmpty()) {
break;
  }
  for (MPartition mpart : partsToDelete) {
dropPartitionCommon(mpart);
  }
}

> Hive error when dropping a table with large number of partitions
> 
>
> Key: HIVE-2907
> URL: https://issues.apache.org/jira/browse/HIVE-2907
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.9.0
> Environment: General. Hive Metastore bug.
>Reporter: Mousom Dhar Gupta
>Assignee: Mousom Dhar Gupta
>Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2907.1.patch.txt, HIVE-2907.2.patch.txt, 
> HIVE-2907.3.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, 
> HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, 
> HIVE-2907.D2505.6.patch, HIVE-2907.D2505.7.patch
>
>   Original Estimate: 10h
>  Remaining Estimate: 10h
>
> Running into an "Out Of Memory" error when trying to drop a table with 128K 
> partitions.
> The methods dropTable in 
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> encounter out of memory errors 
> when dropping tables with lots of partitions because they try to load the 
> metadata for every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3030) escape more chars for script operator

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-3030:
-

Status: Open  (was: Patch Available)

I think we should wait for the tests to get added before committing this patch, 
especially since this modifies a public API (i.e. adding a new configuration 
property).

Also, instead of making this a binary property, would it make more sense to 
allow the user to specify the list of characters that the ScriptOperator should 
escape?


> escape more chars for script operator
> -
>
> Key: HIVE-3030
> URL: https://issues.apache.org/jira/browse/HIVE-3030
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> Only new line was being escaped.
> The same behavior needs to be done for carriage returns, and tabs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3030) escape more chars for script operator

[Siying Dong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280294#comment-13280294
 ] 

Siying Dong commented on HIVE-3030:
---

Logic looks good to me. I'll run unit tests now. In the mean time, can you add 
tests to cover those new cases? Cases like escaping '\', and unescaping cases 
like '\\', ,'\\\t' or '\\\t'?

> escape more chars for script operator
> -
>
> Key: HIVE-3030
> URL: https://issues.apache.org/jira/browse/HIVE-3030
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> Only new line was being escaped.
> The same behavior needs to be done for carriage returns, and tabs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #23

[Apache Jenkins Server]

See 


--
[...truncated 5301 lines...]
 [echo] Project: anttasks

create-dirs:
 [echo] Project: shims
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Warning: 

 does not exist.

init:
 [echo] Project: shims

create-dirs:
 [echo] Project: common
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Copying 2 files to 


init:
 [echo] Project: common

create-dirs:
 [echo] Project: serde
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Warning: 

 does not exist.

init:
 [echo] Project: serde

create-dirs:
 [echo] Project: metastore
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Warning: 

 does not exist.

init:
 [echo] Project: metastore

create-dirs:
 [echo] Project: ql
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created 

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #23

[Apache Jenkins Server]

See 

--
[...truncated 5317 lines...]
[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Warning: 

 does not exist.

init:
 [echo] Project: shims

create-dirs:
 [echo] Project: common
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Copying 2 files to 


init:
 [echo] Project: common

create-dirs:
 [echo] Project: serde
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Warning: 

 does not exist.

init:
 [echo] Project: serde

create-dirs:
 [echo] Project: metastore
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Warning: 

 does not exist.

init:
 [echo] Project: metastore

create-dirs:
 [echo] Project: ql
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

 [copy] Warning: 

 does not exist.

init:
 [echo] Project: ql

create-dirs:
 [echo] Project: contrib
[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[mkdir] Created dir: 

[jira] [Commented] (HIVE-2800) NPE in "create index" without comment clause in external metastore

[Bejoy KS (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280054#comment-13280054
 ] 

Bejoy KS commented on HIVE-2800:


The same bug is found on hive 0.7.1 as well

> NPE in "create index" without comment clause in external metastore
> --
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.7.1, 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Priority: Minor
>
> This happens only when using external metastore (with --hiveconf 
> hive.metastore.uris=thrift://localhost:8088 --hiveconf 
> hive.metastore.local=false). Also if I gave a comment in the statement, this 
> exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred 
> rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - 
> FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at 
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2800) NPE in "create index" without comment clause in external metastore

[Bejoy KS (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bejoy KS updated HIVE-2800:
---

Affects Version/s: 0.7.1

> NPE in "create index" without comment clause in external metastore
> --
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.7.1, 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Priority: Minor
>
> This happens only when using external metastore (with --hiveconf 
> hive.metastore.uris=thrift://localhost:8088 --hiveconf 
> hive.metastore.local=false). Also if I gave a comment in the statement, this 
> exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred 
> rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - 
> FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at 
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira