[jira] Commented: (HIVE-1096) Hive Variables

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870480#action_12870480
 ] 

Edward Capriolo commented on HIVE-1096:
---

I am back on this one. Keep your eye out for the next patch.

> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Attachments: 1096-9.diff, hive-1096-2.diff, hive-1096-7.diff, 
> hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature 
> called "Variables." Basically you can define a variable via command-line 
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via 
> ${DT} within the hive queries. This could be extremely useful. I can't seem 
> to find this feature even on trunk. Is this feature currently anywhere in the 
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is 
> in Driver.compile or Driver.run we can do string substitutions at that level, 
> and further downstream need not be effected. 
> There could be some benefits to doing this further downstream, parser,plan. 
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss 
> this more.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-471) A UDF for simple reflection

[Edward Capriolo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-471:
-

Fix Version/s: 0.6.0
Affects Version/s: 0.5.1
   (was: 0.6.0)

Should be good for trunk.

> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.5.1
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch, 
> HIVE-471.3.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can 
> be invoked with one simple parameter. More complicated functions will require 
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String", 
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1335) DataNucleus should use connection pooling

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870477#action_12870477
 ] 

Edward Capriolo commented on HIVE-1335:
---

Can we go +1?

> DataNucleus should use connection pooling
> -
>
> Key: HIVE-1335
> URL: https://issues.apache.org/jira/browse/HIVE-1335
> Project: Hadoop Hive
>  Issue Type: Improvement
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: commons-dbcp-1.2.2.jar, commons-dbcp.LICENSE, 
> commons-pool-1.2.jar, commons-pool.LICENSE, 
> datanucleus-connectionpool-1.0.2.jar, datanucleus-connectionpool.LICENSE, 
> hive-1335-1.patch.txt, hive-1335-2.patch.txt, hive-1335-3.patch.txt, 
> hive-1335.patch.txt
>
>
> Currently each Data Nucleus operation disconnects and reconnects to the 
> MetaStore over jdbc. Queries fail to even explain properly in cases where a 
> table has many partitions. This is fixed by enabling one parameter and 
> including several jars.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1179) Add UDF array_contains

[Arvind Prabhakar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870475#action_12870475
 ] 

Arvind Prabhakar commented on HIVE-1179:


bq. One minor point - can you make result a member variable of 
GenericUDFArrayContains? This will reduce object creation.

While this will reduce object creation, it will cause correctness problems when 
this UDF is used in an aggregate operation. Using a member variable for 
{{result}} would then mean that all values of aggregated output will always 
reflect the evaluated value of the last row. A similar problem would occur if 
there is a lag between collecting and processing of output values. Hence my 
preference is to keep the implementation as is (stateless).

If you still would like to make it a member variable, please let me know and I 
can make that change. 

> Add UDF array_contains
> --
>
> Key: HIVE-1179
> URL: https://issues.apache.org/jira/browse/HIVE-1179
> Project: Hadoop Hive
>  Issue Type: New Feature
>Reporter: Zheng Shao
>Assignee: Arvind Prabhakar
> Attachments: HIVE-1179-1.patch, HIVE-1179-2.patch, HIVE-1179.patch
>
>
> Returns true or false, depending on whether an element is in an array.
> {{array_contains(T element, array theArray)}}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1179) Add UDF array_contains

[Arvind Prabhakar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arvind Prabhakar updated HIVE-1179:
---

Attachment: HIVE-1179-2.patch

Updated the patch to work with current trunk.

> Add UDF array_contains
> --
>
> Key: HIVE-1179
> URL: https://issues.apache.org/jira/browse/HIVE-1179
> Project: Hadoop Hive
>  Issue Type: New Feature
>Reporter: Zheng Shao
>Assignee: Arvind Prabhakar
> Attachments: HIVE-1179-1.patch, HIVE-1179-2.patch, HIVE-1179.patch
>
>
> Returns true or false, depending on whether an element is in an array.
> {{array_contains(T element, array theArray)}}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Hive-trunk-h0.19 #450

[Apache Hudson Server]

See 

--
[...truncated 13990 lines...]
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_function4.q
[junit] Begin query: unknown_table1.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Out

[jira] Commented: (HIVE-1363) 'SHOW TABLE EXTENDED LIKE' command does not strip single/double quotes

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870365#action_12870365
 ] 

Carl Steinbach commented on HIVE-1363:
--

My understanding is that backticks are reserved for enclosing
identifiers that conflict with other tokens in the HQL grammar, e.g. 

{code}
hive> CREATE TABLE function(stuff INT);
FAILED: Parse Error: line 1:13 mismatched input 'function' expecting Identifier 
in create table statement

hive> CREATE TABLE `function`(stuff INT);
OK

hive> show tables;
OK
function
pokes
{code}

Based on this I would argue that the SHOW TABLES command currently exhibits the 
expected
behavior whereas SHOW TABLE EXTENDED does not:

{code}
hive> SHOW TABLES "pokes*";
OK
pokes

hive> SHOW TABLE EXTENDED LIKE "pokes*";
FAILED: Error in metadata: MetaException(message:Got exception: 
javax.jdo.JDOUserException ')' expected at character 54 in "database.name == 
dbName && ( tableName.matches("(?i)"pokes.*""))")
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask

hive> SHOW TABLES 'pokes*';
OK
pokes

hive> SHOW TABLE EXTENDED LIKE 'pokes*';
OK

hive> SHOW TABLES `pokes*`;
OK

hive> SHOW TABLE EXTENDED LIKE `pokes*`;
OK
tableName:pokes
owner:carl
location:hdfs://localhost/user/hive/warehouse/pokes
inputformat:org.apache.hadoop.mapred.TextInputFormat
outputformat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
columns:struct columns { i32 num}
partitioned:false
partitionColumns:
totalNumberFiles:0
totalFileSize:0
maxFileSize:0
minFileSize:0
lastAccessTime:0
lastUpdateTime:1274517075221

Time taken: 0.579 seconds
hive> 
{code}

> 'SHOW TABLE EXTENDED LIKE' command does not strip single/double quotes
> --
>
> Key: HIVE-1363
> URL: https://issues.apache.org/jira/browse/HIVE-1363
> Project: Hadoop Hive
>  Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0
>
>
> {code}
> hive> SHOW TABLE EXTENDED LIKE pokes;
> OK
> tableName:pokes
> owner:carl
> location:hdfs://localhost/user/hive/warehouse/pokes
> inputformat:org.apache.hadoop.mapred.TextInputFormat
> outputformat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> columns:struct columns { i32 num}
> partitioned:false
> partitionColumns:
> totalNumberFiles:0
> totalFileSize:0
> maxFileSize:0
> minFileSize:0
> lastAccessTime:0
> lastUpdateTime:1274517075221
> hive> SHOW TABLE EXTENDED LIKE "p*";
> FAILED: Error in metadata: MetaException(message:Got exception: 
> javax.jdo.JDOUserException ')' expected at character 54 in "database.name == 
> dbName && ( tableName.matches("(?i)"p.*""))")
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask
> hive> SHOW TABLE EXTENDED LIKE 'p*';
> OK
> hive> SHOW TABLE EXTENDED LIKE `p*`;
> OK
> tableName:pokes
> owner:carl
> location:hdfs://localhost/user/hive/warehouse/pokes
> inputformat:org.apache.hadoop.mapred.TextInputFormat
> outputformat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> columns:struct columns { i32 num}
> partitioned:false
> partitionColumns:
> totalNumberFiles:0
> totalFileSize:0
> maxFileSize:0
> minFileSize:0
> lastAccessTime:0
> lastUpdateTime:1274517075221
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.