[jira] Commented: (HIVE-1455) lateral view does not work with column pruning

[He Yongqiang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888685#action_12888685
 ] 

He Yongqiang commented on HIVE-1455:


testcase loadpart_err failed. Paul, can you take a look?

   [junit] Exception: null
[junit] java.lang.NullPointerException
[junit] at 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_loadpart_err(TestCliDriver.java:100)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at junit.framework.TestCase.runTest(TestCase.java:154)
[junit] at junit.framework.TestCase.runBare(TestCase.java:127)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:106)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] at junit.framework.TestResult.run(TestResult.java:109)
[junit] at junit.framework.TestCase.run(TestCase.java:118)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 19.916 sec
[junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED


> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, 
> hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1466) Add NULL DEFINED AS to ROW FORMAT specification

[Adam Kramer (JIRA)]

Add NULL DEFINED AS to ROW FORMAT specification
---

 Key: HIVE-1466
 URL: https://issues.apache.org/jira/browse/HIVE-1466
 Project: Hadoop Hive
  Issue Type: Improvement
Reporter: Adam Kramer


I just updated the Hive wiki to clarify what some would consider an oddity: 
When NULL values are exported to a script via TRANSFORM, they are converted to 
the string "\N", and then when the script's output is read, any cell that 
contains only \N is treated as a NULL value.

I believe that there are very VERY few reasons why anyone would need cells that 
contain only a backslash and then a capital N to be distinguished from NULL 
cells, but for complete generality, we should allow this.

The way to do that is probably by adding a specification in the ROW FORMAT for 
a table that would allow any string to be treated as a NULL if it is the only 
string in a cell. Some may prefer the empty string, others the word NULL in 
caps, etc. I vote for keeping \N as the default because I am used to it, but 
also for allowing this to be customized.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1385) UDF field() doesn't work

[Siying Dong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siying Dong updated HIVE-1385:
--

Attachment: HIVE-1385.2.patch

use ObjectInspector.compare() and add some test cases.

> UDF field() doesn't work
> 
>
> Key: HIVE-1385
> URL: https://issues.apache.org/jira/browse/HIVE-1385
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Siying Dong
>Assignee: Siying Dong
>Priority: Minor
> Attachments: HIVE-1385.1.patch, HIVE-1385.2.patch
>
>
> I tried it against one of my table:
> hive> desc r;
> OK
> key int
> value string
> a string
> hive> select * from r;
> OK
> 4 val_356 NULL
> 4 val_356 NULL
> 484 val_169 NULL
> 484 val_169 NULL
> 2000 val_169 NULL
> 2000 val_169 NULL
> 3000 val_169 NULL
> 3000 val_169 NULL
> 4000 val_125 NULL
> 4000 val_125 NULL
> hive> select *, field(value, 'val_169') from r; 
> OK
> 4 val_356 NULL 0
> 4 val_356 NULL 0
> 484 val_169 NULL 0
> 484 val_169 NULL 0
> 2000 val_169 NULL 0
> 2000 val_169 NULL 0
> 3000 val_169 NULL 0
> 3000 val_169 NULL 0
> 4000 val_125 NULL 0
> 4000 val_125 NULL 0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-471) A UDF for simple reflection

[Edward Capriolo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-471:
-

Attachment: HIVE-471.4.patch

> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.5.1
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch, 
> HIVE-471.3.patch, HIVE-471.4.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can 
> be invoked with one simple parameter. More complicated functions will require 
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String", 
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-471) A UDF for simple reflection

[Edward Capriolo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-471:
-

Status: Patch Available  (was: Open)

> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.5.1
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch, 
> HIVE-471.3.patch, HIVE-471.4.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can 
> be invoked with one simple parameter. More complicated functions will require 
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String", 
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thrift metastore

[John Sichi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-1428:
-

  Status: Resolved  (was: Patch Available)
Hadoop Flags: [Reviewed]
  Resolution: Fixed

Committed.  Thanks Pradeep!


> ALTER TABLE ADD PARTITION fails with a remote Thrift metastore
> --
>
> Key: HIVE-1428
> URL: https://issues.apache.org/jira/browse/HIVE-1428
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>Assignee: Pradeep Kamath
> Fix For: 0.7.0
>
> Attachments: HIVE-1428-2.patch, HIVE-1428-3.patch, HIVE-1428.patch, 
> TestHiveMetaStoreRemote.java
>
>
> If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD 
> PARTITION commands will fail with an error similar to the following:
> [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE 
> mytable add partition(datestamp = '20091101', srcid = '10',action) location 
> '/user/pradeepk/mytable/20091101/10';"
> 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found 
> in the classpath. Usage of hadoop-site.xml is deprecated. Instead use 
> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of 
> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
> Hive history 
> file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
> FAILED: Error in metadata: org.apache.thrift.TApplicationException: 
> get_partition failed: unknown result
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask
> [prade...@chargesize:~/dev/howl]
> This is due to a check that tries to retrieve the partition to see if it 
> exists. If it does not, an attempt is made to pass a null value from the 
> metastore. Since thrift does not support null return values, an exception is 
> thrown.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1385) UDF field() doesn't work

[He Yongqiang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888633#action_12888633
 ] 

He Yongqiang commented on HIVE-1385:


discussed with Siying offline.
1. use ObjectInspectUtils.compare
2. add more test, the first argument can also be a column
3. add a negative test for complex types.

> UDF field() doesn't work
> 
>
> Key: HIVE-1385
> URL: https://issues.apache.org/jira/browse/HIVE-1385
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Siying Dong
>Assignee: Siying Dong
>Priority: Minor
> Attachments: HIVE-1385.1.patch
>
>
> I tried it against one of my table:
> hive> desc r;
> OK
> key int
> value string
> a string
> hive> select * from r;
> OK
> 4 val_356 NULL
> 4 val_356 NULL
> 484 val_169 NULL
> 484 val_169 NULL
> 2000 val_169 NULL
> 2000 val_169 NULL
> 3000 val_169 NULL
> 3000 val_169 NULL
> 4000 val_125 NULL
> 4000 val_125 NULL
> hive> select *, field(value, 'val_169') from r; 
> OK
> 4 val_356 NULL 0
> 4 val_356 NULL 0
> 484 val_169 NULL 0
> 484 val_169 NULL 0
> 2000 val_169 NULL 0
> 2000 val_169 NULL 0
> 3000 val_169 NULL 0
> 3000 val_169 NULL 0
> 4000 val_125 NULL 0
> 4000 val_125 NULL 0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1455) lateral view does not work with column pruning

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-1455:


Attachment: HIVE-1455.4.patch

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, 
> hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1455) lateral view does not work with column pruning

[He Yongqiang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888636#action_12888636
 ] 

He Yongqiang commented on HIVE-1455:


running test now.

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, 
> hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1455) lateral view does not work with column pruning

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-1455:


Attachment: (was: HIVE-1455.4.patch)

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, 
> hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1455) lateral view does not work with column pruning

[He Yongqiang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888632#action_12888632
 ] 

He Yongqiang commented on HIVE-1455:


overall looks good. please remove the LOG.

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, 
> hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1385) UDF field() doesn't work

[Siying Dong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siying Dong updated HIVE-1385:
--

Status: Patch Available  (was: Open)

> UDF field() doesn't work
> 
>
> Key: HIVE-1385
> URL: https://issues.apache.org/jira/browse/HIVE-1385
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Siying Dong
>Assignee: Siying Dong
>Priority: Minor
> Attachments: HIVE-1385.1.patch
>
>
> I tried it against one of my table:
> hive> desc r;
> OK
> key int
> value string
> a string
> hive> select * from r;
> OK
> 4 val_356 NULL
> 4 val_356 NULL
> 484 val_169 NULL
> 484 val_169 NULL
> 2000 val_169 NULL
> 2000 val_169 NULL
> 3000 val_169 NULL
> 3000 val_169 NULL
> 4000 val_125 NULL
> 4000 val_125 NULL
> hive> select *, field(value, 'val_169') from r; 
> OK
> 4 val_356 NULL 0
> 4 val_356 NULL 0
> 484 val_169 NULL 0
> 484 val_169 NULL 0
> 2000 val_169 NULL 0
> 2000 val_169 NULL 0
> 3000 val_169 NULL 0
> 3000 val_169 NULL 0
> 4000 val_125 NULL 0
> 4000 val_125 NULL 0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1455) lateral view does not work with column pruning

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-1455:


Attachment: HIVE-1455.4.patch

Added an additional test case where we do a lateral view on a subquery.

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, 
> hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1385) UDF field() doesn't work

[He Yongqiang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888615#action_12888615
 ] 

He Yongqiang commented on HIVE-1385:


will take a look.

> UDF field() doesn't work
> 
>
> Key: HIVE-1385
> URL: https://issues.apache.org/jira/browse/HIVE-1385
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Siying Dong
>Assignee: Siying Dong
>Priority: Minor
> Attachments: HIVE-1385.1.patch
>
>
> I tried it against one of my table:
> hive> desc r;
> OK
> key int
> value string
> a string
> hive> select * from r;
> OK
> 4 val_356 NULL
> 4 val_356 NULL
> 484 val_169 NULL
> 484 val_169 NULL
> 2000 val_169 NULL
> 2000 val_169 NULL
> 3000 val_169 NULL
> 3000 val_169 NULL
> 4000 val_125 NULL
> 4000 val_125 NULL
> hive> select *, field(value, 'val_169') from r; 
> OK
> 4 val_356 NULL 0
> 4 val_356 NULL 0
> 484 val_169 NULL 0
> 484 val_169 NULL 0
> 2000 val_169 NULL 0
> 2000 val_169 NULL 0
> 3000 val_169 NULL 0
> 3000 val_169 NULL 0
> 4000 val_125 NULL 0
> 4000 val_125 NULL 0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1455) lateral view does not work with column pruning

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-1455:


Attachment: (was: HIVE-1455.3.patch)

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1455) lateral view does not work with column pruning

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-1455:


Attachment: HIVE-1455.3.patch

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (HIVE-1455) lateral view does not work with column pruning

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang reassigned HIVE-1455:
---

Assignee: Paul Yang  (was: He Yongqiang)

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: Paul Yang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1455) lateral view does not work with column pruning

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-1455:


Attachment: HIVE-1455.3.patch

This patch modifies the column pruner to create a special case for the lateral 
view join operator. This will result in more pruned columns. 

> lateral view does not work with column pruning 
> ---
>
> Key: HIVE-1455
> URL: https://issues.apache.org/jira/browse/HIVE-1455
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Hive-trunk-h0.19 #496

[Apache Hudson Server]

See 

Changes:

[nzhang] HIVE-1462. Report progress in FileSinkOperator for multiple directory 
case (Siying Dong via Ning Zhang)

[heyongqiang] HIVE-1056. Predicate push down does not work with UDTF's (Paul 
Yang via He Yongqiang)

[jvs] HIVE-1438. SENTENCES() UDF for natural language tokenization
(Mayank Lahiri via jvs)

--
[...truncated 13926 lines...]
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_function4.q
[junit] Begin query: unknown_table1.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOO

[jira] Created: (HIVE-1465) hive-site.xml ${user.name} not replaced for local-file derby metastore connection URL

[Carl Steinbach (JIRA)]

hive-site.xml ${user.name} not replaced for local-file derby metastore 
connection URL
-

 Key: HIVE-1465
 URL: https://issues.apache.org/jira/browse/HIVE-1465
 Project: Hadoop Hive
  Issue Type: Bug
  Components: Configuration
Affects Versions: 0.5.0, 0.6.0
Reporter: Carl Steinbach


Seems that for this parameter

{code}

javax.jdo.option.ConnectionURL
jdbc:derby:;databaseName=/var/lib/hive/metastore/${user.name}_db;create=true
JDBC connect string for a JDBC metastore

{code}

$\{user.name\} is never replaced by the actual user name:

{code}
$ ls -la /var/lib/hive/metastore/
total 24
drwxrwxrwt 3 root root 4096 Apr 30 12:37 .
drwxr-xr-x 3 root root 4096 Apr 30 12:25 ..
drwxrwxr-x 5 hadoop hadoop 4096 Apr 30 12:37 ${user.name}_db
{code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1408) add option to let hive automatically run in local mode based on tunable heuristics

[Joydeep Sen Sarma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joydeep Sen Sarma updated HIVE-1408:


Attachment: 1408.2.patch
1408.2.q.out.patch

v2. this is ready for review.

added tests:
- the tests now use 'pfile:///' namespace as the default warehouse filesystem. 
This is served by a proxy filesystem class that passes requests to the local 
file system
- this comprehensively tests all the file system issues related to running in 
local mode (where there is now a difference between the intermediate data's 
file system and the warehouse's file system). there are several small bug fixes 
related to bugs discovered because of this test mode.
- there are changes in a lot of test results as a result of the new namespace 
as well as because of the changes in tmp file naming. i am attaching a extra 
diff (.q.out.patch) that shows only the interesting changes.
- some tests have been modified to run with a non-local setting for the 
jobtracker and with auto-local-mode turned on. this tests the new functionality.
- there is one test (archive.q) that's still breaking because of the filesystem 
issues. waiting for a fix from pyang. but it should not stop the review.

> add option to let hive automatically run in local mode based on tunable 
> heuristics
> --
>
> Key: HIVE-1408
> URL: https://issues.apache.org/jira/browse/HIVE-1408
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Joydeep Sen Sarma
>Assignee: Joydeep Sen Sarma
> Attachments: 1408.1.patch, 1408.2.patch, 1408.2.q.out.patch
>
>
> as a followup to HIVE-543 - we should have a simple option (enabled by 
> default) to let hive run in local mode if possible.
> two levels of options are desirable:
> 1. hive.exec.mode.local.auto=true/false // control whether local mode is 
> automatically chosen
> 2. Options to control different heuristics, some naiive examples:
>  hive.exec.mode.local.auto.input.size.max=1G // don't choose local mode 
> if data > 1G
>  hive.exec.mode.local.auto.script.enable=true/false // choose if local 
> mode is enabled for queries with user scripts
> this can be implemented as a pre/post execution hook. It makes sense to 
> provide this as a standard hook in the hive codebase since it's likely to 
> improve response time for many users (especially for test queries).
> the initial proposal is to choose this at a query level and not at per 
> hive-task (ie. hadoop job) level. per job-level requires more changes to 
> compilation (to not pre-commit to hdfs or local scratch directories at 
> compile time).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1126) Missing some Jdbc functionality like getTables getColumns and HiveResultSet.get* methods based on column name.

[Bennie Schut (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888365#action_12888365
 ] 

Bennie Schut commented on HIVE-1126:


Hi Vinithra,

I just started working on it again. I've had little time to work on this but in 
the end we also need this committed. But if I run into a problem I'll probably 
ask for some help.

Thanks.

> Missing some Jdbc functionality like getTables getColumns and 
> HiveResultSet.get* methods based on column name.
> --
>
> Key: HIVE-1126
> URL: https://issues.apache.org/jira/browse/HIVE-1126
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Clients
>Affects Versions: 0.6.0
>Reporter: Bennie Schut
>Assignee: Bennie Schut
>Priority: Minor
> Attachments: HIVE-1126-1.patch, HIVE-1126.patch, 
> HIVE-1126_patch(0.5.0_source).patch
>
>
> I've been using the hive jdbc driver more and more and was missing some 
> functionality which I added
> HiveDatabaseMetaData.getTables
> Using "show tables" to get the info from hive.
> HiveDatabaseMetaData.getColumns
> Using "describe tablename" to get the columns.
> This makes using something like SQuirreL a lot nicer since you have the list 
> of tables and just click on the content tab to see what's in the table.
> I also implemented
> HiveResultSet.getObject(String columnName) so you call most get* methods 
> based on the column name.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.