[jira] Updated: (HIVE-1413) bring a table/partition offline

2010-08-03 Thread Siying Dong (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siying Dong updated HIVE-1413:
--

Attachment: HIVE-1413.2.patch

1. use parameters without making any change to metastore
2. add support of no_drop mode. Make every mode like bitmap instead of levels
3. cover the case of dropping tables

> bring a table/partition offline
> ---
>
> Key: HIVE-1413
> URL: https://issues.apache.org/jira/browse/HIVE-1413
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
> Fix For: 0.7.0
>
> Attachments: HIVE-1413.1.patch, HIVE-1413.2.patch
>
>
> There should be a way to bring a table/partition offline.
> At that time, no read/write operations should be supported on that table.
> It would be very useful for housekeeping operations

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1413) bring a table/partition offline

2010-08-03 Thread Siying Dong (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siying Dong updated HIVE-1413:
--

Attachment: (was: HIVE-1414.1.patch)

> bring a table/partition offline
> ---
>
> Key: HIVE-1413
> URL: https://issues.apache.org/jira/browse/HIVE-1413
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
> Fix For: 0.7.0
>
> Attachments: HIVE-1413.1.patch
>
>
> There should be a way to bring a table/partition offline.
> At that time, no read/write operations should be supported on that table.
> It would be very useful for housekeeping operations

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path

2010-08-03 Thread He Yongqiang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1510:
---

Attachment: hive-1510.1.patch

> HiveCombineInputFormat should not use prefix matching to find the 
> partitionDesc for a given path
> 
>
> Key: HIVE-1510
> URL: https://issues.apache.org/jira/browse/HIVE-1510
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: hive-1510.1.patch
>
>
> set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
> drop table combine_3_srcpart_seq_rc;
> create table combine_3_srcpart_seq_rc (key int , value string) partitioned by 
> (ds string, hr string) stored as sequencefile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="00") select * from src;
> alter table combine_3_srcpart_seq_rc set fileformat rcfile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="001") select * from src;
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="00");
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="001");
> select * from combine_3_srcpart_seq_rc where ds="2010-08-03" order by key;
> drop table combine_3_srcpart_seq_rc;
> will fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path

2010-08-03 Thread He Yongqiang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1510:
---

Status: Patch Available  (was: Open)

> HiveCombineInputFormat should not use prefix matching to find the 
> partitionDesc for a given path
> 
>
> Key: HIVE-1510
> URL: https://issues.apache.org/jira/browse/HIVE-1510
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: hive-1510.1.patch
>
>
> set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
> drop table combine_3_srcpart_seq_rc;
> create table combine_3_srcpart_seq_rc (key int , value string) partitioned by 
> (ds string, hr string) stored as sequencefile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="00") select * from src;
> alter table combine_3_srcpart_seq_rc set fileformat rcfile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="001") select * from src;
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="00");
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="001");
> select * from combine_3_srcpart_seq_rc where ds="2010-08-03" order by key;
> drop table combine_3_srcpart_seq_rc;
> will fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path

2010-08-03 Thread He Yongqiang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1510:
---

Attachment: hive-1510.1.patch

> HiveCombineInputFormat should not use prefix matching to find the 
> partitionDesc for a given path
> 
>
> Key: HIVE-1510
> URL: https://issues.apache.org/jira/browse/HIVE-1510
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
>
> set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
> drop table combine_3_srcpart_seq_rc;
> create table combine_3_srcpart_seq_rc (key int , value string) partitioned by 
> (ds string, hr string) stored as sequencefile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="00") select * from src;
> alter table combine_3_srcpart_seq_rc set fileformat rcfile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="001") select * from src;
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="00");
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="001");
> select * from combine_3_srcpart_seq_rc where ds="2010-08-03" order by key;
> drop table combine_3_srcpart_seq_rc;
> will fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path

2010-08-03 Thread He Yongqiang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1510:
---

Attachment: (was: hive-1510.1.patch)

> HiveCombineInputFormat should not use prefix matching to find the 
> partitionDesc for a given path
> 
>
> Key: HIVE-1510
> URL: https://issues.apache.org/jira/browse/HIVE-1510
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
>
> set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
> drop table combine_3_srcpart_seq_rc;
> create table combine_3_srcpart_seq_rc (key int , value string) partitioned by 
> (ds string, hr string) stored as sequencefile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="00") select * from src;
> alter table combine_3_srcpart_seq_rc set fileformat rcfile;
> insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
> hr="001") select * from src;
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="00");
> desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="001");
> select * from combine_3_srcpart_seq_rc where ds="2010-08-03" order by key;
> drop table combine_3_srcpart_seq_rc;
> will fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path

2010-08-03 Thread He Yongqiang (JIRA) 
HiveCombineInputFormat should not use prefix matching to find the partitionDesc 
for a given path


 Key: HIVE-1510
 URL: https://issues.apache.org/jira/browse/HIVE-1510
 Project: Hadoop Hive
  Issue Type: Bug
Reporter: He Yongqiang
Assignee: He Yongqiang


set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;

drop table combine_3_srcpart_seq_rc;

create table combine_3_srcpart_seq_rc (key int , value string) partitioned by 
(ds string, hr string) stored as sequencefile;

insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
hr="00") select * from src;

alter table combine_3_srcpart_seq_rc set fileformat rcfile;
insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", 
hr="001") select * from src;

desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="00");
desc extended combine_3_srcpart_seq_rc partition(ds="2010-08-03", hr="001");

select * from combine_3_srcpart_seq_rc where ds="2010-08-03" order by key;

drop table combine_3_srcpart_seq_rc;


will fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1509) Monitor the working set of the number of files

2010-08-03 Thread Namit Jain (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895106#action_12895106
 ] 

Namit Jain commented on HIVE-1509:
--

With dynamic partitions, the number of intermediate files can grow very fast.

For example, consider a query with 10,000 mappers and 100 files per mapper - it 
can create up to 1 million files before merging them at the end.
The cluster may be down by the time the query finishes.

It is a good idea to track the number of files through a counter, and kill the 
query if the number exceeds a given threshold


> Monitor the working set of the number of files 
> ---
>
> Key: HIVE-1509
> URL: https://issues.apache.org/jira/browse/HIVE-1509
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.7.0
>Reporter: Namit Jain
>Assignee: Ning Zhang
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1509) Monitor the working set of the number of files

2010-08-03 Thread Namit Jain (JIRA) 
Monitor the working set of the number of files 
---

 Key: HIVE-1509
 URL: https://issues.apache.org/jira/browse/HIVE-1509
 Project: Hadoop Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.7.0
Reporter: Namit Jain
Assignee: Ning Zhang




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1434) Cassandra Storage Handler

2010-08-03 Thread Edward Capriolo (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-1434:
--

Attachment: hive-1434-2-patch.txt

Closing in on this one. This patch sets up build environment correctly. Proper 
test infrastructure. Patch is much cleaner. Still working on 
Serializing/Deserialing correctly so not very functional. 80% I think.

> Cassandra Storage Handler
> -
>
> Key: HIVE-1434
> URL: https://issues.apache.org/jira/browse/HIVE-1434
> Project: Hadoop Hive
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Attachments: cas-handle.tar.gz, hive-1434-1.txt, hive-1434-2-patch.txt
>
>
> Add a cassandra storage handler.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Hudson build is back to normal : Hive-trunk-h0.20 #336

2010-08-03 Thread Apache Hudson Server 
See

Build failed in Hudson: Hive-trunk-h0.19 #513

2010-08-03 Thread Apache Hudson Server 
See 

Changes:

[jvs] HIVE-1414. Automatically invoke .hiverc init script
(Edward Capriolo via jvs)

[nzhang] HIVE-1422 (2nd trial). skip counter update when 
RunningJob.getCounters() returns null (Joydeep Sen Sarma via Ning Zhang)

[jvs] HIVE-1126 cleanup noted in HIVE-1504 (remove HiveResultSet.java).

--
[...truncated 13372 lines...]
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Copying data from 

[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Copying data from 

[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Copying data from 

[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Copying data from 

[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Copying data from 

[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Copying data from 

[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] PO

[jira] Commented: (HIVE-558) describe extended table/partition output is cryptic

2010-08-03 Thread Thiruvel Thirumoolan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895002#action_12895002
 ] 

Thiruvel Thirumoolan commented on HIVE-558:
---

Can I pick this one up if someone else isnt working on it? 

I would approach it by writing a Utils function to parse and display output of 
metastore/src/gen-javabean/*Table.toString() in a proper format. Or is there a 
Thrift helper that gets this done easier?

> describe extended table/partition output is cryptic
> ---
>
> Key: HIVE-558
> URL: https://issues.apache.org/jira/browse/HIVE-558
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Prasad Chakka
>
> describe extended table prints out the Thrift metadata object directly. The 
> information from it is not easy to read or parse. Output should be easily 
> read and can be simple parsed to get table location etc by programs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1378) Return value for map, array, or UDF (that returns map/array) needs to return a string

2010-08-03 Thread Jerome Boulon (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerome Boulon updated HIVE-1378:


Assignee: (was: Jerome Boulon)

> Return value for map, array, or UDF (that returns map/array) needs to return 
> a string 
> --
>
> Key: HIVE-1378
> URL: https://issues.apache.org/jira/browse/HIVE-1378
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Drivers
>Reporter: Jerome Boulon
>
> In order to be able to select/display any data from JDBC Hive driver, return 
> value for map, array, or UDF (that returns map/array) needs to return a string

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (HIVE-1378) Return value for map, array, or UDF (that returns map/array) needs to return a string

2010-08-03 Thread Carl Steinbach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach reassigned HIVE-1378:


Assignee: Jerome Boulon  (was: Carl Steinbach)

> Return value for map, array, or UDF (that returns map/array) needs to return 
> a string 
> --
>
> Key: HIVE-1378
> URL: https://issues.apache.org/jira/browse/HIVE-1378
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Drivers
>Reporter: Jerome Boulon
>Assignee: Jerome Boulon
>
> In order to be able to select/display any data from JDBC Hive driver, return 
> value for map, array, or UDF (that returns map/array) needs to return a string

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (HIVE-1493) incorrect explanation when local mode not chosen automatically

2010-08-03 Thread Joydeep Sen Sarma (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joydeep Sen Sarma resolved HIVE-1493.
-

  Assignee: Joydeep Sen Sarma
Resolution: Fixed

fixed via HIVE-1422

> incorrect explanation when local mode not chosen automatically
> --
>
> Key: HIVE-1493
> URL: https://issues.apache.org/jira/browse/HIVE-1493
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Joydeep Sen Sarma
>Assignee: Joydeep Sen Sarma
>Priority: Minor
>
> slipped past in 1408:
> // check for max input size   
>   
> if (inputSummary.getLength() > maxBytes)
> return "Input Size (= " + maxBytes + ") is larger than " +
> HiveConf.ConfVars.LOCALMODEMAXBYTES.varname + " (= " + maxBytes + 
> ")";
> printing same value twice.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.