date:20140729


[ 
https://issues.apache.org/jira/browse/HIVE-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077445#comment-14077445
 ] 

Hive QA commented on HIVE-7535:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12658342/HIVE-7535.2.patch

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5787 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/88/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/88/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-88/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12658342

 Make use of number of nulls column statistics in filter rule
 

 Key: HIVE-7535
 URL: https://issues.apache.org/jira/browse/HIVE-7535
 Project: Hive
  Issue Type: Sub-task
  Components: Query Processor, Statistics
Affects Versions: 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
Priority: Minor
 Fix For: 0.13.0

 Attachments: HIVE-7535.1.patch, HIVE-7535.2.patch


 The filter rule does not make use of number of nulls column statistics for 
 IS NULL and IS NOT NULL expression evaluation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7532) allow disabling direct sql per query with external metastore


 [ 
https://issues.apache.org/jira/browse/HIVE-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7532:


Attachment: HIVE-7532.1.patch.txt

 allow disabling direct sql per query with external metastore
 

 Key: HIVE-7532
 URL: https://issues.apache.org/jira/browse/HIVE-7532
 Project: Hive
  Issue Type: Improvement
Reporter: Sergey Shelukhin
 Attachments: HIVE-7532.1.patch.txt


 Currently with external metastore, direct sql can only be disabled via 
 metastore config globally. Perhaps it makes sense to have the ability to 
 propagate the setting per query from client to override the metastore 
 setting, e.g. if one particular query causes it to fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7532) allow disabling direct sql per query with external metastore


 [ 
https://issues.apache.org/jira/browse/HIVE-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7532:


Assignee: Navis
  Status: Patch Available  (was: Open)

 allow disabling direct sql per query with external metastore
 

 Key: HIVE-7532
 URL: https://issues.apache.org/jira/browse/HIVE-7532
 Project: Hive
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Navis
 Attachments: HIVE-7532.1.patch.txt


 Currently with external metastore, direct sql can only be disabled via 
 metastore config globally. Perhaps it makes sense to have the ability to 
 propagate the setting per query from client to override the metastore 
 setting, e.g. if one particular query causes it to fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6601) alter database commands should support schema synonym keyword


 [ 
https://issues.apache.org/jira/browse/HIVE-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-6601:


   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks for the review, Thejas.

 alter database commands should support schema synonym keyword
 -

 Key: HIVE-6601
 URL: https://issues.apache.org/jira/browse/HIVE-6601
 Project: Hive
  Issue Type: Bug
Reporter: Thejas M Nair
Assignee: Navis
 Fix For: 0.14.0

 Attachments: HIVE-6601.1.patch.txt


 It should be possible to use alter schema  as an alternative to alter 
 database.  But the syntax is not currently supported.
 {code}
 alter schema db1 set owner user x;  
 NoViableAltException(215@[])
 FAILED: ParseException line 1:6 cannot recognize input near 'schema' 'db1' 
 'set' in alter statement
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7539) streaming windowing UDAF seems to be broken without Partition Spec


 [ 
https://issues.apache.org/jira/browse/HIVE-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7539:


Attachment: HIVE-7539.1.patch.txt

 streaming windowing UDAF seems to be broken without Partition Spec
 --

 Key: HIVE-7539
 URL: https://issues.apache.org/jira/browse/HIVE-7539
 Project: Hive
  Issue Type: Bug
Reporter: Laljo John Pullokkaran
Assignee: Harish Butani
 Attachments: HIVE-7539.1.patch.txt


 {code:sql}
 select  avg(c_int) over(rows between 1 PRECEDING and current row) from t1
 {code}
 results in 
 {noformat}
 :1}}
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:271)
   ... 9 more
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:180)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStreamingEvaluator$SumAvgEnhancer.iterate(GenericUDAFStreamingEvaluator.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:185)
   at 
 org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:348)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:318)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.processOp(PTFOperator.java:131)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:262)
   ... 9 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7539) streaming windowing UDAF seems to be broken without Partition Spec


[ 
https://issues.apache.org/jira/browse/HIVE-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077466#comment-14077466
 ] 

Navis commented on HIVE-7539:
-

[~jpullokkaran] Could you try the patch attached?

 streaming windowing UDAF seems to be broken without Partition Spec
 --

 Key: HIVE-7539
 URL: https://issues.apache.org/jira/browse/HIVE-7539
 Project: Hive
  Issue Type: Bug
Reporter: Laljo John Pullokkaran
Assignee: Harish Butani
 Attachments: HIVE-7539.1.patch.txt


 {code:sql}
 select  avg(c_int) over(rows between 1 PRECEDING and current row) from t1
 {code}
 results in 
 {noformat}
 :1}}
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:271)
   ... 9 more
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:180)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStreamingEvaluator$SumAvgEnhancer.iterate(GenericUDAFStreamingEvaluator.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:185)
   at 
 org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:348)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:318)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.processOp(PTFOperator.java:131)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:262)
   ... 9 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 24043: DefaultHiveAuthorizationProvider should not initialize a new HiveConf

2014-07-29 Thread Navis Ryu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24043/
---

Review request for hive.


Bugs: HIVE-6437
https://issues.apache.org/jira/browse/HIVE-6437


Repository: hive-git


Description
---

During a HS2 connection, every SessionState got initializes a new 
DefaultHiveAuthorizationProvider object (on stock configs).

In turn, DefaultHiveAuthorizationProvider carries a {{new HiveConf(…)}} that 
may prove too expensive, and unnecessary to do, since SessionState itself sends 
in a fully applied HiveConf to it in the first place.


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ae7cd53 
  
contrib/src/java/org/apache/hadoop/hive/contrib/metastore/hooks/TestURLHook.java
 39562ea 
  contrib/src/test/queries/clientnegative/url_hook.q c346432 
  contrib/src/test/queries/clientpositive/url_hook.q PRE-CREATION 
  contrib/src/test/results/clientnegative/url_hook.q.out 601fd93 
  contrib/src/test/results/clientpositive/url_hook.q.out PRE-CREATION 
  data/conf/hive-site.xml fe8080a 
  itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java 
e8d405d 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java
 0bb022e 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 2fefa06 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
5cc1cd8 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
d26183b 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
5add436 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStoreProxy.java 
1cf09d4 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 81323f6 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/DefaultHiveAuthorizationProvider.java
 2fa512c 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/StorageBasedAuthorizationProvider.java
 0dfd997 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveRoleGrant.java
 ce07f32 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java
 ce12edb 
  ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java d218271 
  ql/src/test/queries/clientnegative/authorization_cannot_create_all_role.q 
de91e91 
  ql/src/test/queries/clientnegative/authorization_cannot_create_default_role.q 
42a42f6 
  ql/src/test/queries/clientnegative/authorization_cannot_create_none_role.q 
0d14cde 
  ql/src/test/queries/clientnegative/authorization_caseinsensitivity.q d5ea284 
  ql/src/test/queries/clientnegative/authorization_drop_db_cascade.q edeae9b 
  ql/src/test/queries/clientnegative/authorization_drop_db_empty.q 46d4d0f 
  ql/src/test/queries/clientnegative/authorization_drop_role_no_admin.q a7aa17f 
  ql/src/test/queries/clientnegative/authorization_priv_current_role_neg.q 
463358a 
  ql/src/test/queries/clientnegative/authorization_role_cycles1.q a819d20 
  ql/src/test/queries/clientnegative/authorization_role_cycles2.q 423f030 
  ql/src/test/queries/clientnegative/authorization_role_grant.q c5c500a 
  ql/src/test/queries/clientnegative/authorization_role_grant2.q 7fdf157 
  ql/src/test/queries/clientnegative/authorization_role_grant_nosuchrole.q 
f456165 
  ql/src/test/queries/clientnegative/authorization_role_grant_otherrole.q 
f91abdb 
  ql/src/test/queries/clientnegative/authorization_role_grant_otheruser.q 
a530043 
  ql/src/test/queries/clientnegative/authorization_rolehierarchy_privs.q 
d9f4c7c 
  ql/src/test/queries/clientnegative/authorization_set_role_neg2.q 03f748f 
  ql/src/test/queries/clientnegative/authorization_show_grant_otherrole.q 
a709d16 
  ql/src/test/queries/clientnegative/authorization_show_grant_otheruser_all.q 
2073cda 
  
ql/src/test/queries/clientnegative/authorization_show_grant_otheruser_alltabs.q 
672b81b 
  ql/src/test/queries/clientnegative/authorization_show_grant_otheruser_wtab.q 
7d95a9d 
  ql/src/test/queries/clientpositive/authorization_1_sql_std.q 381937c 
  ql/src/test/queries/clientpositive/authorization_admin_almighty1.q 45c4a7d 
  ql/src/test/queries/clientpositive/authorization_admin_almighty2.q ce99670 
  ql/src/test/queries/clientpositive/authorization_create_func1.q 65a7b33 
  ql/src/test/queries/clientpositive/authorization_create_macro1.q fb60500 
  ql/src/test/queries/clientpositive/authorization_insert.q 6cce469 
  ql/src/test/queries/clientpositive/authorization_owner_actions_db.q 36ab260 
  ql/src/test/queries/clientpositive/authorization_role_grant1.q c062ef2 
  ql/src/test/queries/clientpositive/authorization_role_grant2.q 34e19a2 
  ql/src/test/queries/clientpositive/authorization_set_show_current_role.q 
6b5af6e 
  ql/src/test/queries/clientpositive/authorization_show_grant.q 5f7a33b 
  ql/src/test/queries/clientpositive/authorization_view_sqlstd.q

[jira] [Commented] (HIVE-6806) CREATE TABLE should support STORED AS AVRO


[ 
https://issues.apache.org/jira/browse/HIVE-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077482#comment-14077482
 ] 

Lefty Leverenz commented on HIVE-6806:
--

[~singhashish], wikidoc updates are handled various ways.  Sometimes the 
developer takes care of it, sometimes I write it up and ask for review, and 
occasionally someone else writes it up.  I usually edit what others write, 
making sure version information is included and cross references get made.

* In this case, I think I could revise the DDL sections adequately but you 
would probably do a better job revising the Avro SerDe wiki, keeping in mind 
that the old information needs to remain for users of previous releases.

* If you don't want to do it yourself, or even if you do, a release note on 
this JIRA ticket would be an excellent start.

* If you have an in-house tech writer who could do the job, their contribution 
would be most welcome --- my backlog of doc tasks is daunting, I'm just doing 
this as a retirement hobby, and it's summertime. ;)

The links in my previous comment show where revisions should go.  The Avro 
SerDe doc has several examples of CREATE TABLE --- the simplest approach is to 
add a second example for each one, saying In Hive 0.14.0 and later, this 
syntax can be used: (or something similar).  A general statement or discussion 
of the new syntax would be good too.

Since the 0.14.0 release is a few months away, the documentation could wait 
although it might be best done while still fresh in your mind.  To gain write 
access to the wiki, follow the instructions here:

* [About This Wiki | 
https://cwiki.apache.org/confluence/display/Hive/AboutThisWiki]

 CREATE TABLE should support STORED AS AVRO
 --

 Key: HIVE-6806
 URL: https://issues.apache.org/jira/browse/HIVE-6806
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Affects Versions: 0.12.0
Reporter: Jeremy Beard
Assignee: Ashish Kumar Singh
Priority: Minor
  Labels: Avro, TODOC14
 Fix For: 0.14.0

 Attachments: HIVE-6806.1.patch, HIVE-6806.2.patch, HIVE-6806.3.patch, 
 HIVE-6806.patch


 Avro is well established and widely used within Hive, however creating 
 Avro-backed tables requires the messy listing of the SerDe, InputFormat and 
 OutputFormat classes.
 Similarly to HIVE-5783 for Parquet, Hive would be easier to use if it had 
 native Avro support.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6601) alter database commands should support schema synonym keyword


 [ 
https://issues.apache.org/jira/browse/HIVE-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lefty Leverenz updated HIVE-6601:
-

Labels: TODOC14  (was: )

 alter database commands should support schema synonym keyword
 -

 Key: HIVE-6601
 URL: https://issues.apache.org/jira/browse/HIVE-6601
 Project: Hive
  Issue Type: Bug
Reporter: Thejas M Nair
Assignee: Navis
  Labels: TODOC14
 Fix For: 0.14.0

 Attachments: HIVE-6601.1.patch.txt


 It should be possible to use alter schema  as an alternative to alter 
 database.  But the syntax is not currently supported.
 {code}
 alter schema db1 set owner user x;  
 NoViableAltException(215@[])
 FAILED: ParseException line 1:6 cannot recognize input near 'schema' 'db1' 
 'set' in alter statement
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7529) insert query fails on hdfs federation + viewfs still exists


[ 
https://issues.apache.org/jira/browse/HIVE-7529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077493#comment-14077493
 ] 

Hive QA commented on HIVE-7529:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12658341/HIVE-7529.1.patch.txt

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5787 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/89/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/89/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-89/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12658341

 insert query fails on hdfs federation + viewfs still exists
 -

 Key: HIVE-7529
 URL: https://issues.apache.org/jira/browse/HIVE-7529
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.13.1
Reporter: John
Assignee: Navis
 Attachments: HIVE-7529.1.patch.txt


 $ echo 111,222 /tmp/testtable
 $ sudo -u hive hive
 hive create table test (a int, b int) row format delimited fields terminated 
 by ',' stored as textfile;
 OK
 Time taken: 2.355 seconds
 hive load data local inpath '/tmp/testtable' overwrite into table test;



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6601) alter database commands should support schema synonym keyword


[ 
https://issues.apache.org/jira/browse/HIVE-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077495#comment-14077495
 ] 

Lefty Leverenz commented on HIVE-6601:
--

Documentation goes here:

* [DDL -- Alter Database | 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterDatabase]

similar to create and drop database here (although it would be better to use 
new lines of syntax to show the version information, instead of | syntax):

* [DDL -- Create/Drop/Alter Database | 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/AlterDatabase]

 alter database commands should support schema synonym keyword
 -

 Key: HIVE-6601
 URL: https://issues.apache.org/jira/browse/HIVE-6601
 Project: Hive
  Issue Type: Bug
Reporter: Thejas M Nair
Assignee: Navis
  Labels: TODOC14
 Fix For: 0.14.0

 Attachments: HIVE-6601.1.patch.txt


 It should be possible to use alter schema  as an alternative to alter 
 database.  But the syntax is not currently supported.
 {code}
 alter schema db1 set owner user x;  
 NoViableAltException(215@[])
 FAILED: ParseException line 1:6 cannot recognize input near 'schema' 'db1' 
 'set' in alter statement
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7497) Fix some default values in HiveConf


 [ 
https://issues.apache.org/jira/browse/HIVE-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7497:


   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks for the contribution, Dong Chen. 

 Fix some default values in HiveConf
 ---

 Key: HIVE-7497
 URL: https://issues.apache.org/jira/browse/HIVE-7497
 Project: Hive
  Issue Type: Task
Reporter: Brock Noland
Assignee: Dong Chen
 Fix For: 0.14.0

 Attachments: HIVE-7497.1.patch, HIVE-7497.patch


 HIVE-5160 resolves an env variable at runtime via calling System.getenv(). As 
 long as the variable is not defined when you run the build null is returned 
 and the path is not placed in the hive-default,template. However if it is 
 defined it will populate hive-default.template with a path which will be 
 different based on the user running the build. We should use 
 $\{system:HIVE_CONF_DIR\} instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 24044: insert query fails on hdfs federation + viewfs still exists

2014-07-29 Thread Navis Ryu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24044/
---

Review request for hive.


Bugs: HIVE-7529
https://issues.apache.org/jira/browse/HIVE-7529


Repository: hive-git


Description
---

$ echo 111,222 /tmp/testtable
$ sudo -u hive hive
hive create table test (a int, b int) row format delimited fields terminated 
by ',' stored as textfile;
OK
Time taken: 2.355 seconds
hive load data local inpath '/tmp/testtable' overwrite into table test;


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/Context.java 74b4711 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 7d62f45 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 8604c5e 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 77f56c1 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java c0322fb 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
83b6304 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d38270c 

Diff: https://reviews.apache.org/r/24044/diff/


Testing
---


Thanks,

Navis Ryu

Review Request 24045: beeline always exits with 0 status, should exit with non-zero status on error

2014-07-29 Thread Navis Ryu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24045/
---

Review request for hive.


Bugs: HIVE-6978
https://issues.apache.org/jira/browse/HIVE-6978


Repository: hive-git


Description
---

Was supposed to be fixed in Hive 0.12 (HIVE-4364). Doesn't look fixed from here.

[i@p sqoop]$ beeline -u 'jdbc:hive2://p:1/k;principal=hive/p@L' -e select 
* from MEMBERS --outputformat=vertical
scan complete in 3ms
Connecting to jdbc:hive2://p:1/k;principal=hive/p@L
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/avro/avro-tools-1.7.5-cdh5.0.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Connected to: Apache Hive (version 0.12.0-cdh5.0.0)
Driver: Hive JDBC (version 0.12.0-cdh5.0.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
-hiveconf (No such file or directory)
hive.aux.jars.path=[redacted]
Error: Error while compiling statement: FAILED: SemanticException [Error 
10001]: Line 1:14 Table not found 'MEMBERS' (state=42S02,code=10001)
Beeline version 0.12.0-cdh5.0.0 by Apache Hive
Closing: org.apache.hive.jdbc.HiveConnection
[inter@p sqoop]$ echo $?
0


Diffs
-

  beeline/src/java/org/apache/hive/beeline/AbstractCommandHandler.java c762096 
  beeline/src/java/org/apache/hive/beeline/BeeLine.java 10fd2e2 
  beeline/src/java/org/apache/hive/beeline/CommandHandler.java 0441bce 
  beeline/src/java/org/apache/hive/beeline/ReflectiveCommandHandler.java 
0fb636e 
  beeline/src/test/org/apache/hive/beeline/TestBeelineArgParsing.java 8329db8 

Diff: https://reviews.apache.org/r/24045/diff/


Testing
---


Thanks,

Navis Ryu

[jira] [Updated] (HIVE-7068) Integrate AccumuloStorageHandler


 [ 
https://issues.apache.org/jira/browse/HIVE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7068:


Assignee: Josh Elser

 Integrate AccumuloStorageHandler
 

 Key: HIVE-7068
 URL: https://issues.apache.org/jira/browse/HIVE-7068
 Project: Hive
  Issue Type: New Feature
Reporter: Josh Elser
Assignee: Josh Elser
 Fix For: 0.14.0

 Attachments: HIVE-7068.1.patch


 [Accumulo|http://accumulo.apache.org] is a BigTable-clone which is similar to 
 HBase. Some [initial 
 work|https://github.com/bfemiano/accumulo-hive-storage-manager] has been done 
 to support querying an Accumulo table using Hive already. It is not a 
 complete solution as, most notably, the current implementation presently 
 lacks support for INSERTs.
 I would like to polish up the AccumuloStorageHandler (presently based on 
 0.10), implement missing basic functionality and compare it to the 
 HBaseStorageHandler (to ensure that we follow the same general usage 
 patterns).
 I've also been in communication with [~bfem] (the initial author) who 
 expressed interest in working on this again. I hope to coordinate efforts 
 with him.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7068) Integrate AccumuloStorageHandler


[ 
https://issues.apache.org/jira/browse/HIVE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077506#comment-14077506
 ] 

Navis commented on HIVE-7068:
-

It's really big deal, but I don't know anything on accumulo. Can we get this in 
first and see what will happen?

 Integrate AccumuloStorageHandler
 

 Key: HIVE-7068
 URL: https://issues.apache.org/jira/browse/HIVE-7068
 Project: Hive
  Issue Type: New Feature
Reporter: Josh Elser
 Fix For: 0.14.0

 Attachments: HIVE-7068.1.patch


 [Accumulo|http://accumulo.apache.org] is a BigTable-clone which is similar to 
 HBase. Some [initial 
 work|https://github.com/bfemiano/accumulo-hive-storage-manager] has been done 
 to support querying an Accumulo table using Hive already. It is not a 
 complete solution as, most notably, the current implementation presently 
 lacks support for INSERTs.
 I would like to polish up the AccumuloStorageHandler (presently based on 
 0.10), implement missing basic functionality and compare it to the 
 HBaseStorageHandler (to ensure that we follow the same general usage 
 patterns).
 I've also been in communication with [~bfem] (the initial author) who 
 expressed interest in working on this again. I hope to coordinate efforts 
 with him.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7497) Fix some default values in HiveConf


 [ 
https://issues.apache.org/jira/browse/HIVE-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lefty Leverenz updated HIVE-7497:
-

Labels: TODOC14  (was: )

 Fix some default values in HiveConf
 ---

 Key: HIVE-7497
 URL: https://issues.apache.org/jira/browse/HIVE-7497
 Project: Hive
  Issue Type: Task
Reporter: Brock Noland
Assignee: Dong Chen
  Labels: TODOC14
 Fix For: 0.14.0

 Attachments: HIVE-7497.1.patch, HIVE-7497.patch


 HIVE-5160 resolves an env variable at runtime via calling System.getenv(). As 
 long as the variable is not defined when you run the build null is returned 
 and the path is not placed in the hive-default,template. However if it is 
 defined it will populate hive-default.template with a path which will be 
 different based on the user running the build. We should use 
 $\{system:HIVE_CONF_DIR\} instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7532) allow disabling direct sql per query with external metastore


[ 
https://issues.apache.org/jira/browse/HIVE-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077530#comment-14077530
 ] 

Hive QA commented on HIVE-7532:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12658369/HIVE-7532.1.patch.txt

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 5787 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.metastore.TestMetastoreVersion.testVersionMisMatch
org.apache.hadoop.hive.metastore.TestMetastoreVersion.testVersionRestriction
org.apache.hive.jdbc.TestSSL.testConnectionMismatch
org.apache.hive.jdbc.TestSSL.testSSLConnectionWithProperty
org.apache.hive.jdbc.TestSSL.testSSLConnectionWithURL
org.apache.hive.jdbc.TestSSL.testSSLFetch
org.apache.hive.jdbc.TestSSL.testSSLFetchHttp
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/90/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/90/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-90/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12658369

 allow disabling direct sql per query with external metastore
 

 Key: HIVE-7532
 URL: https://issues.apache.org/jira/browse/HIVE-7532
 Project: Hive
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Navis
 Attachments: HIVE-7532.1.patch.txt


 Currently with external metastore, direct sql can only be disabled via 
 metastore config globally. Perhaps it makes sense to have the ability to 
 propagate the setting per query from client to override the metastore 
 setting, e.g. if one particular query causes it to fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7497) Fix some default values in HiveConf


[ 
https://issues.apache.org/jira/browse/HIVE-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077536#comment-14077536
 ] 

Lefty Leverenz commented on HIVE-7497:
--

This renames configuration parameter *hive.global.init.file.location* (created 
in HIVE-5160) to *hive.server2.global.init.file.location* and revises its 
description, as well as changing its default from 
System.getenv(HIVE_CONF_DIR) to $\{env:HIVE_CONF_DIR\} and changing the 
default of configuration parameter *hive.hwi.war.file* from 
$\{system:HWI_WAR_FILE\} to $\{env:HWI_WAR_FILE\}.  

*hive.server2.global.init.file.location* needs to be documented in two places, 
with version information and a link to this JIRA ticket:

* [Hive Web Interface -- Configuration | 
https://cwiki.apache.org/confluence/display/Hive/HiveWebInterface#HiveWebInterface-Configuration]
* [Configuration Properties -- Hive Web Interface (HWI) | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveWebInterface(HWI)]

But it's not clear to me whether the default value changes need any user doc 
updates -- do they?

 Fix some default values in HiveConf
 ---

 Key: HIVE-7497
 URL: https://issues.apache.org/jira/browse/HIVE-7497
 Project: Hive
  Issue Type: Task
Reporter: Brock Noland
Assignee: Dong Chen
  Labels: TODOC14
 Fix For: 0.14.0

 Attachments: HIVE-7497.1.patch, HIVE-7497.patch


 HIVE-5160 resolves an env variable at runtime via calling System.getenv(). As 
 long as the variable is not defined when you run the build null is returned 
 and the path is not placed in the hive-default,template. However if it is 
 defined it will populate hive-default.template with a path which will be 
 different based on the user running the build. We should use 
 $\{system:HIVE_CONF_DIR\} instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-1183) hive.hwi.war.file vanished from hive-default.xml. error message prints wrong variable. does not start by default.


[ 
https://issues.apache.org/jira/browse/HIVE-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077550#comment-14077550
 ] 

Lefty Leverenz commented on HIVE-1183:
--

The wiki had given $\{HIVE_HOME\}/lib/hive_hwi.war as the default for 
*hive.hwi.war.file* (note the underscore) so I changed it to  
$\{HIVE_HOME\}/lib/hive-hwi-version.war in the Hive Web Interface doc and 
gave both defaults with version information in the Configuration Properties doc.

* [Hive Web Interface -- Configuration | 
https://cwiki.apache.org/confluence/display/Hive/HiveWebInterface#HiveWebInterface-Configuration]
* [Configuration Properties -- Hive Web Interface | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveWebInterface(HWI)]

 hive.hwi.war.file vanished from hive-default.xml. error message prints wrong 
 variable. does not start by default.
 -

 Key: HIVE-1183
 URL: https://issues.apache.org/jira/browse/HIVE-1183
 Project: Hive
  Issue Type: Bug
  Components: Configuration
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Blocker
 Fix For: 0.5.0

 Attachments: HIVE-1183.2.b-0.5_and_trunk.diff, 
 HIVE-1183.showidea.diff, hive-1183-b5.0.diff, hive-1183-trunk.diff, 
 hive-1183.diff


 hive.hwi.war.file vanished from hive-default.xml.
 error message prints wrong variable.
 since the full path is no longer supplied. The old documentation no longer 
 applies
  String hwiWAR = conf.getVar(HiveConf.ConfVars.HIVEHWIWARFILE);
  String hivehome = System.getenv().get(HIVE_HOME);
File hwiWARFile = new File(hivehome, hwiWAR);
if (!hwiWARFile.exists()) {
  l4j.fatal(HWI WAR file not found at  + hwiWAR);
  System.exit(1);
}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7338) Create SparkPlanGenerator


 [ 
https://issues.apache.org/jira/browse/HIVE-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-7338:
--

Attachment: HIVE-7338.patch

 Create SparkPlanGenerator
 -

 Key: HIVE-7338
 URL: https://issues.apache.org/jira/browse/HIVE-7338
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-7338.patch


 Translate SparkWork into SparkPlan. The translation may be invoked by 
 SparkClient when executing SparkTask.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7338) Create SparkPlanGenerator


 [ 
https://issues.apache.org/jira/browse/HIVE-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-7338:
--

Status: Patch Available  (was: Open)

 Create SparkPlanGenerator
 -

 Key: HIVE-7338
 URL: https://issues.apache.org/jira/browse/HIVE-7338
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-7338.patch


 Translate SparkWork into SparkPlan. The translation may be invoked by 
 SparkClient when executing SparkTask.
 NO PRECOMMIT TESTS. This is for spark branch only.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7338) Create SparkPlanGenerator


 [ 
https://issues.apache.org/jira/browse/HIVE-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-7338:
--

Description: 
Translate SparkWork into SparkPlan. The translation may be invoked by 
SparkClient when executing SparkTask.

NO PRECOMMIT TESTS. This is for spark branch only.

  was:Translate SparkWork into SparkPlan. The translation may be invoked by 
SparkClient when executing SparkTask.


 Create SparkPlanGenerator
 -

 Key: HIVE-7338
 URL: https://issues.apache.org/jira/browse/HIVE-7338
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-7338.patch


 Translate SparkWork into SparkPlan. The translation may be invoked by 
 SparkClient when executing SparkTask.
 NO PRECOMMIT TESTS. This is for spark branch only.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7338) Create SparkPlanGenerator


 [ 
https://issues.apache.org/jira/browse/HIVE-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-7338:
--

Component/s: Spark

 Create SparkPlanGenerator
 -

 Key: HIVE-7338
 URL: https://issues.apache.org/jira/browse/HIVE-7338
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-7338.patch


 Translate SparkWork into SparkPlan. The translation may be invoked by 
 SparkClient when executing SparkTask.
 NO PRECOMMIT TESTS. This is for spark branch only.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

2014-07-29 Thread Rui Li (JIRA)

Rui Li created HIVE-7540:


 Summary: NotSerializableException encountered when using sortByKey 
transformation
 Key: HIVE-7540
 URL: https://issues.apache.org/jira/browse/HIVE-7540
 Project: Hive
  Issue Type: Bug
  Components: Spark
 Environment: Spark-1.0.1
Reporter: Rui Li


This exception is thrown when sortByKey is used as the shuffle transformation 
between MapWork and ReduceWork:
{quote}
org.apache.spark.SparkException: Job aborted due to stage failure: Task not 
serializable: java.io.NotSerializableException: 
org.apache.hadoop.io.BytesWritable
at 
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1049)
at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1033)
at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1031)
at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at 
org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1031)
at 
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitMissingTasks(DAGScheduler.scala:772)
at 
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:715)
at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$submitStage$4.apply(DAGScheduler.scala:719)
at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$submitStage$4.apply(DAGScheduler.scala:718)
at scala.collection.immutable.List.foreach(List.scala:318)
at 
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:718)
at 
org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:699)
…
{quote}

 The root cause is that the RangePartitioner used by sortByKey contains 
rangeBounds: Array[BytesWritable], which is considered not serializable in 
spark.
A workaround to this issue is to set the number of partitions to 1 when calling 
sortByKey, in which case the rangeBounds will be just an empty array.

NO PRECOMMIT TESTS. This is for spark branch only.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7338) Create SparkPlanGenerator


 [ 
https://issues.apache.org/jira/browse/HIVE-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-7338:
--

   Resolution: Fixed
Fix Version/s: spark-branch
   Status: Resolved  (was: Patch Available)

Patch committed to spark branch.

 Create SparkPlanGenerator
 -

 Key: HIVE-7338
 URL: https://issues.apache.org/jira/browse/HIVE-7338
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: spark-branch

 Attachments: HIVE-7338.patch


 Translate SparkWork into SparkPlan. The translation may be invoked by 
 SparkClient when executing SparkTask.
 NO PRECOMMIT TESTS. This is for spark branch only.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-7335) Create SparkPlan, DAG representation of a Spark job


 [ 
https://issues.apache.org/jira/browse/HIVE-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang resolved HIVE-7335.
---

   Resolution: Fixed
Fix Version/s: spark-branch

Fixed as part of HIVE-7338. Will create followup JIRAs for remaining work.

 Create SparkPlan, DAG representation of a Spark job
 ---

 Key: HIVE-7335
 URL: https://issues.apache.org/jira/browse/HIVE-7335
 Project: Hive
  Issue Type: Sub-task
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: spark-branch


 Encapsulate RDD, MapFunction, ReduceFunction, and SparkShuffler in a graph 
 representation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7330) Create SparkTask


[ 
https://issues.apache.org/jira/browse/HIVE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077585#comment-14077585
 ] 

Xuefu Zhang commented on HIVE-7330:
---

[~chinnalalam] Hope you don't mind, but I'm assigning this to Na, as we need to 
make progress on this.

 Create SparkTask
 

 Key: HIVE-7330
 URL: https://issues.apache.org/jira/browse/HIVE-7330
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Chinna Rao Lalam

 SparkTask handles the execution of SparkWork. It will execute a graph of map 
 and reduce work using a SparkClient instance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (HIVE-7330) Create SparkTask


[ 
https://issues.apache.org/jira/browse/HIVE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077585#comment-14077585
 ] 

Xuefu Zhang edited comment on HIVE-7330 at 7/29/14 10:27 AM:
-

[~chinnalalam], I'm wondering if you're working on this. Hope you don't mind, 
but I'm assigning this to [~nyang] as we need to make progress on this. Let me 
know if otherwise.


was (Author: xuefuz):
[~chinnalalam], I'm wondering if you're working on this. Hope you don't mind, 
but I'm assigning this to Na as we need to make progress on this. Let me know 
if otherwise.

 Create SparkTask
 

 Key: HIVE-7330
 URL: https://issues.apache.org/jira/browse/HIVE-7330
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Chinna Rao Lalam

 SparkTask handles the execution of SparkWork. It will execute a graph of map 
 and reduce work using a SparkClient instance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (HIVE-7330) Create SparkTask


[ 
https://issues.apache.org/jira/browse/HIVE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077585#comment-14077585
 ] 

Xuefu Zhang edited comment on HIVE-7330 at 7/29/14 10:26 AM:
-

[~chinnalalam], I'm wondering if you're working on this. Hope you don't mind, 
but I'm assigning this to Na as we need to make progress on this. Let me know 
if otherwise.


was (Author: xuefuz):
[~chinnalalam] Hope you don't mind, but I'm assigning this to Na, as we need to 
make progress on this.

 Create SparkTask
 

 Key: HIVE-7330
 URL: https://issues.apache.org/jira/browse/HIVE-7330
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Chinna Rao Lalam

 SparkTask handles the execution of SparkWork. It will execute a graph of map 
 and reduce work using a SparkClient instance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7330) Create SparkTask


[ 
https://issues.apache.org/jira/browse/HIVE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077589#comment-14077589
 ] 

Xuefu Zhang commented on HIVE-7330:
---

[~nyang] If Chinna is fine with this, please plan to work on this. The basic 
work is done, but we'd like to put it in a production quality. Please take 
TezTask as a reference.

 Create SparkTask
 

 Key: HIVE-7330
 URL: https://issues.apache.org/jira/browse/HIVE-7330
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Chinna Rao Lalam

 SparkTask handles the execution of SparkWork. It will execute a graph of map 
 and reduce work using a SparkClient instance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7541) Support union all on Spark

Xuefu Zhang created HIVE-7541:
-

 Summary: Support union all on Spark
 Key: HIVE-7541
 URL: https://issues.apache.org/jira/browse/HIVE-7541
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang


For union all operator, we will use Spark's union transformation. Refer to the 
design doc on wiki for more information.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7327) Refactoring: make Hive map side data processing reusable


[ 
https://issues.apache.org/jira/browse/HIVE-7327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077693#comment-14077693
 ] 

Xuefu Zhang commented on HIVE-7327:
---

It seems it's easier to use ExecMapper directly than any refactoring. Postpone 
this item for now for later consideration.

 Refactoring: make Hive map side data processing reusable
 

 Key: HIVE-7327
 URL: https://issues.apache.org/jira/browse/HIVE-7327
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang

 ExecMapper is Hive's mapper implementation for MapReduce. Table rows are read 
 by MR framework and processed by ExecMapper.map() method, which invokes 
 Hive's map-side operator tree starting from MapOperator. This task is to 
 extract the map-side data processing offered by the operator tree so that it 
 can be used by other execution engine such as Spark. This is purely 
 refactoring the existing code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7328) Refactoring: make Hive reduce side data processing reusable


[ 
https://issues.apache.org/jira/browse/HIVE-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077694#comment-14077694
 ] 

Xuefu Zhang commented on HIVE-7328:
---

It seems it's easier to use ExecReducer directly than any refactoring. Postpone 
this item for now for later consideration.

 Refactoring: make Hive reduce side data processing reusable
 ---

 Key: HIVE-7328
 URL: https://issues.apache.org/jira/browse/HIVE-7328
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang

 ExecReducer is Hive's reducer implementation for MapReduce. Table rows are 
 shuffled by MR framework to ExecReducer and further processed by 
 ExecReducer.reduce() method, which invokes Hive's reduce-side operator tree 
 starting. This task is to extract the reduce-side data processing offered by 
 the operator tree so that it can be reused by other execution engine such as 
 Spark. This is purely refactoring the existing code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7539) streaming windowing UDAF seems to be broken without Partition Spec

2014-07-29 Thread Harish Butani (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077818#comment-14077818
 ] 

Harish Butani commented on HIVE-7539:
-

+1 lgtm

 streaming windowing UDAF seems to be broken without Partition Spec
 --

 Key: HIVE-7539
 URL: https://issues.apache.org/jira/browse/HIVE-7539
 Project: Hive
  Issue Type: Bug
Reporter: Laljo John Pullokkaran
Assignee: Harish Butani
 Attachments: HIVE-7539.1.patch.txt


 {code:sql}
 select  avg(c_int) over(rows between 1 PRECEDING and current row) from t1
 {code}
 results in 
 {noformat}
 :1}}
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:271)
   ... 9 more
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:180)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStreamingEvaluator$SumAvgEnhancer.iterate(GenericUDAFStreamingEvaluator.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:185)
   at 
 org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:348)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:318)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.processOp(PTFOperator.java:131)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:262)
   ... 9 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7539) streaming windowing UDAF seems to be broken without Partition Spec


 [ 
https://issues.apache.org/jira/browse/HIVE-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-7539:
---

Status: Patch Available  (was: Open)

 streaming windowing UDAF seems to be broken without Partition Spec
 --

 Key: HIVE-7539
 URL: https://issues.apache.org/jira/browse/HIVE-7539
 Project: Hive
  Issue Type: Bug
Reporter: Laljo John Pullokkaran
Assignee: Harish Butani
 Attachments: HIVE-7539.1.patch.txt


 {code:sql}
 select  avg(c_int) over(rows between 1 PRECEDING and current row) from t1
 {code}
 results in 
 {noformat}
 :1}}
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:271)
   ... 9 more
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:180)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStreamingEvaluator$SumAvgEnhancer.iterate(GenericUDAFStreamingEvaluator.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:185)
   at 
 org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:348)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:318)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.processOp(PTFOperator.java:131)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:262)
   ... 9 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6437) DefaultHiveAuthorizationProvider should not initialize a new HiveConf


[ 
https://issues.apache.org/jira/browse/HIVE-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077853#comment-14077853
 ] 

Ashutosh Chauhan commented on HIVE-6437:


This seems like a useful change. [~thejas] would you like to review this one?

 DefaultHiveAuthorizationProvider should not initialize a new HiveConf
 -

 Key: HIVE-6437
 URL: https://issues.apache.org/jira/browse/HIVE-6437
 Project: Hive
  Issue Type: Bug
  Components: Configuration
Affects Versions: 0.13.0
Reporter: Harsh J
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-6437.1.patch.txt, HIVE-6437.2.patch.txt, 
 HIVE-6437.3.patch.txt, HIVE-6437.4.patch.txt, HIVE-6437.5.patch.txt


 During a HS2 connection, every SessionState got initializes a new 
 DefaultHiveAuthorizationProvider object (on stock configs).
 In turn, DefaultHiveAuthorizationProvider carries a {{new HiveConf(…)}} that 
 may prove too expensive, and unnecessary to do, since SessionState itself 
 sends in a fully applied HiveConf to it in the first place.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7330) Create SparkTask

2014-07-29 Thread Na Yang (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077937#comment-14077937
 ] 

Na Yang commented on HIVE-7330:
---

Xuefu, I will start to work on this task. Please re-assign it to me if Chinna 
is fine with this. Thanks.  

 Create SparkTask
 

 Key: HIVE-7330
 URL: https://issues.apache.org/jira/browse/HIVE-7330
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Chinna Rao Lalam

 SparkTask handles the execution of SparkWork. It will execute a graph of map 
 and reduce work using a SparkClient instance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7529) insert query fails on hdfs federation + viewfs still exists


[ 
https://issues.apache.org/jira/browse/HIVE-7529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077958#comment-14077958
 ] 

Ashutosh Chauhan commented on HIVE-7529:


+1

 insert query fails on hdfs federation + viewfs still exists
 -

 Key: HIVE-7529
 URL: https://issues.apache.org/jira/browse/HIVE-7529
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.13.1
Reporter: John
Assignee: Navis
 Attachments: HIVE-7529.1.patch.txt


 $ echo 111,222 /tmp/testtable
 $ sudo -u hive hive
 hive create table test (a int, b int) row format delimited fields terminated 
 by ',' stored as textfile;
 OK
 Time taken: 2.355 seconds
 hive load data local inpath '/tmp/testtable' overwrite into table test;



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-3017) hive-exec jar, contains classes from other modules(hive-serde, hive-shims, hive-common etc) duplicating those classes in two jars

2014-07-29 Thread murali namburi (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077969#comment-14077969
 ] 

murali namburi commented on HIVE-3017:
--

I suggest to have hive-ql jar which has related to hive-ql and hive-exec could 
have ql and shade jars(jackson,thrift..etc)

In one of our app, we have latest jackson which has conflicts to hive-exec jar 
(shade jackson classes) , our app doesn't work if we include hive-exec, also we 
cannot remove from dependency as it is shaded jar. we have remove the jackson 
class from the jar and manually pack it make them available to the app, so that 
app is working as expected



 hive-exec jar, contains classes from other modules(hive-serde, hive-shims, 
 hive-common etc) duplicating those classes in two jars
 -

 Key: HIVE-3017
 URL: https://issues.apache.org/jira/browse/HIVE-3017
 Project: Hive
  Issue Type: Bug
Reporter: Jakob Homan

 HIVE-2646 added the jars from hive-serde to the hive-exec class:
 {noformat}
 ...
  0 Wed May 09 20:56:30 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/
   1971 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/ListTypeInfo.class
   2396 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/MapTypeInfo.class
   2788 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/PrimitiveTypeInfo.class
   4408 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/StructTypeInfo.class
900 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.class
   6576 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.class
   1231 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$1.class
   1239 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser$Token.class
   7145 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser.class
  14482 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.class
   2594 Wed May 09 20:56:28 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/UnionTypeInfo.class
144 Wed May 09 20:56:30 PDT 2012 
 org/apache/hadoop/hive/serde2/typeinfo/package-info.class
 ...{noformat}
 Was this intentional? If so, the serde jar should be deprecated. If not, the 
 serde classes should be removed since this creates two sources of truth for 
 them and can cause other problems (see HCATALOG-407).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7249) HiveTxnManager.closeTxnManger() throws if called after commitTxn()


[ 
https://issues.apache.org/jira/browse/HIVE-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078005#comment-14078005
 ] 

Ashutosh Chauhan commented on HIVE-7249:


+1

 HiveTxnManager.closeTxnManger() throws if called after commitTxn()
 --

 Key: HIVE-7249
 URL: https://issues.apache.org/jira/browse/HIVE-7249
 Project: Hive
  Issue Type: Bug
  Components: Locking
Affects Versions: 0.13.1
Reporter: Eugene Koifman
Assignee: Alan Gates
 Attachments: HIVE-7249.patch


  I openTxn() and acquireLocks() for a query that looks like INSERT INTO T 
 PARTITION(p) SELECT * FROM T.
 Then I call commitTxn().  Then I call closeTxnManger() I get an exception 
 saying lock not found (the only lock in this txn).  So it seems TxnMgr 
 doesn't know that commit released the locks.
 Here is the stack trace and some log output which maybe useful:
 {noformat}
 2014-06-17 15:54:40,771 DEBUG mapreduce.TransactionContext 
 (TransactionContext.java:onCommitJob(128)) - 
 onCommitJob(job_local557130041_0001). this=46719652
 2014-06-17 15:54:40,771 DEBUG lockmgr.DbTxnManager 
 (DbTxnManager.java:commitTxn(205)) - Committing txn 1
 2014-06-17 15:54:40,771 DEBUG txn.TxnHandler (TxnHandler.java:getDbTime(872)) 
 - Going to execute query values current_timestamp
 2014-06-17 15:54:40,772 DEBUG txn.TxnHandler 
 (TxnHandler.java:heartbeatTxn(1423)) - Going to execute query select 
 txn_state from TXNS where txn_id = 1 for\
  update
 2014-06-17 15:54:40,773 DEBUG txn.TxnHandler 
 (TxnHandler.java:heartbeatTxn(1438)) - Going to execute update update TXNS 
 set txn_last_heartbeat = 140304568\
 0772 where txn_id = 1
 2014-06-17 15:54:40,778 DEBUG txn.TxnHandler 
 (TxnHandler.java:heartbeatTxn(1440)) - Going to commit
 2014-06-17 15:54:40,779 DEBUG txn.TxnHandler (TxnHandler.java:commitTxn(344)) 
 - Going to execute insert insert into COMPLETED_TXN_COMPONENTS select tc_txn\
 id, tc_database, tc_table, tc_partition from TXN_COMPONENTS where tc_txnid = 
 1
 2014-06-17 15:54:40,784 DEBUG txn.TxnHandler (TxnHandler.java:commitTxn(352)) 
 - Going to execute update delete from TXN_COMPONENTS where tc_txnid = 1
 2014-06-17 15:54:40,788 DEBUG txn.TxnHandler (TxnHandler.java:commitTxn(356)) 
 - Going to execute update delete from HIVE_LOCKS where hl_txnid = 1
 2014-06-17 15:54:40,791 DEBUG txn.TxnHandler (TxnHandler.java:commitTxn(359)) 
 - Going to execute update delete from TXNS where txn_id = 1
 2014-06-17 15:54:40,794 DEBUG txn.TxnHandler (TxnHandler.java:commitTxn(361)) 
 - Going to commit
 2014-06-17 15:54:40,795 WARN  mapreduce.TransactionContext 
 (TransactionContext.java:cleanup(317)) - 
 cleanupJob(JobID=job_local557130041_0001)this=46719652
 2014-06-17 15:54:40,795 DEBUG lockmgr.DbLockManager 
 (DbLockManager.java:unlock(109)) - Unlocking id:1
 2014-06-17 15:54:40,796 DEBUG txn.TxnHandler (TxnHandler.java:getDbTime(872)) 
 - Going to execute query values current_timestamp
 2014-06-17 15:54:40,796 DEBUG txn.TxnHandler 
 (TxnHandler.java:heartbeatLock(1402)) - Going to execute update update 
 HIVE_LOCKS set hl_last_heartbeat = 140\
 3045680796 where hl_lock_ext_id = 1
 2014-06-17 15:54:40,800 DEBUG txn.TxnHandler 
 (TxnHandler.java:heartbeatLock(1405)) - Going to rollback
 2014-06-17 15:54:40,804 ERROR metastore.RetryingHMSHandler 
 (RetryingHMSHandler.java:invoke(143)) - NoSuchLockException(message:No such 
 lock: 1)
 at 
 org.apache.hadoop.hive.metastore.txn.TxnHandler.heartbeatLock(TxnHandler.java:1407)
 at 
 org.apache.hadoop.hive.metastore.txn.TxnHandler.unlock(TxnHandler.java:477)
 at 
 org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.unlock(HiveMetaStore.java:4817)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at 
 org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
 at com.sun.proxy.$Proxy14.unlock(Unknown Source)
 at 
 org.apache.hadoop.hive.metastore.HiveMetaStoreClient.unlock(HiveMetaStoreClient.java:1598)
 at 
 org.apache.hadoop.hive.ql.lockmgr.DbLockManager.unlock(DbLockManager.java:110)
 at 
 org.apache.hadoop.hive.ql.lockmgr.DbLockManager.close(DbLockManager.java:162)
 at 
 org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.destruct(DbTxnManager.java:300)
 at 
 org.apache.hadoop.hive.ql.lockmgr.HiveTxnManagerImpl.closeTxnManager(HiveTxnManagerImpl.java:39)
 at 
 org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.closeTxnManager(DbTxnManager.java:43)
 at

[jira] [Commented] (HIVE-7246) Hive transaction manager hardwires bonecp as the JDBC pooling implementation


[ 
https://issues.apache.org/jira/browse/HIVE-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077999#comment-14077999
 ] 

Ashutosh Chauhan commented on HIVE-7246:


+1

 Hive transaction manager hardwires bonecp as the JDBC pooling implementation
 

 Key: HIVE-7246
 URL: https://issues.apache.org/jira/browse/HIVE-7246
 Project: Hive
  Issue Type: Bug
  Components: Transactions
Affects Versions: 0.13.0
Reporter: Alan Gates
Assignee: Alan Gates
 Attachments: HIVE-7246-2.patch, HIVE-7246.patch


 Currently TxnManager hardwires BoneCP as the JDBC connection pooling 
 implementation.  Instead it should use the same connection pooling that the 
 metastore does.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7525) Research to find out if it's possible to submit Spark jobs concurrently using shared SparkContext

2014-07-29 Thread Chao (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-7525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078056#comment-14078056
 ] 

Chao commented on HIVE-7525:


I modified SparkClient to make it submit rdd4 via a separate thread, which 
simply does the foreach in the run method. However, I keep getting this 
issue
about not being able to find the plan file:

14/07/29 10:01:37 INFO exec.Utilities: local path = 
hdfs://localhost:8020/tmp/hive-chao/6ab5877a-ba1a-4761-971e-45d9b46cd3c6/hive_2014-07-29_10-01-28_749_8375059517503664847-1/-mr-10003/1a80d789-63d8-43bb-b3f4-4ad74a66b0af/map.xml
14/07/29 10:01:37 INFO exec.Utilities: Open file to read in plan: 
hdfs://localhost:8020/tmp/hive-chao/6ab5877a-ba1a-4761-971e-45d9b46cd3c6/hive_2014-07-29_10-01-28_749_8375059517503664847-1/-mr-10003/1a80d789-63d8-43bb-b3f4-4ad74a66b0af/map.xml
14/07/29 10:01:37 INFO exec.Utilities: File not found: File does not exist: 
/tmp/hive-chao/6ab5877a-ba1a-4761-971e-45d9b46cd3c6/hive_2014-07-29_10-01-28_749_8375059517503664847-1/-mr-10003/1a80d789-63d8-43bb-b3f4-4ad74a66b0af/map.xml
at 
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:65)
at 
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:55)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1726)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1669)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1649)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1621)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:482)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:322)
at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1986)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1982)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1980)

On the other hand, if I trigger the foreach in the current thread, everything 
is fine.
Maybe it's because the hadoop FS doesn't allow accessing the same file from 
different threads? Not sure why.

 Research to find out if it's possible to submit Spark jobs concurrently using 
 shared SparkContext
 -

 Key: HIVE-7525
 URL: https://issues.apache.org/jira/browse/HIVE-7525
 Project: Hive
  Issue Type: Task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Chao

 Refer to HIVE-7503 and SPARK-2688. Find out if it's possible to submit 
 multiple spark jobs concurrently using a shared SparkContext. SparkClient's 
 code can be manipulated for this test. Here is the process:
 1. Transform rdd1 to rdd2 using some transformation.
 2. call rdd2.cache() to persist it in memory.
 3. in two threads, calling accordingly:
 Thread a. rdd2 - rdd3; rdd3.foreach()
 Thread b. rdd2 - rdd4; rdd4.foreach()
 It would be nice to find out monitoring and error reporting aspects.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7488) pass column names being used for inputs to authorization api

2014-07-29 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-7488:


Attachment: HIVE-7488.5.patch

HIVE-7488.5.patch - rebased again


 pass column names being used for inputs to authorization api
 

 Key: HIVE-7488
 URL: https://issues.apache.org/jira/browse/HIVE-7488
 Project: Hive
  Issue Type: Bug
  Components: Authorization
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Attachments: HIVE-7488.1.patch, HIVE-7488.2.patch, 
 HIVE-7488.3.patch.txt, HIVE-7488.4.patch, HIVE-7488.5.patch


 HivePrivilegeObject in the authorization api has support for columns, but the 
 columns being used are not being populated for non grant-revoke queries.
 This is for enabling any implementation of the api to use this column 
 information for its authorization decisions.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7539) streaming windowing UDAF seems to be broken without Partition Spec


[ 
https://issues.apache.org/jira/browse/HIVE-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078149#comment-14078149
 ] 

Hive QA commented on HIVE-7539:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12658373/HIVE-7539.1.patch.txt

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5772 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/91/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/91/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-91/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12658373

 streaming windowing UDAF seems to be broken without Partition Spec
 --

 Key: HIVE-7539
 URL: https://issues.apache.org/jira/browse/HIVE-7539
 Project: Hive
  Issue Type: Bug
Reporter: Laljo John Pullokkaran
Assignee: Harish Butani
 Attachments: HIVE-7539.1.patch.txt


 {code:sql}
 select  avg(c_int) over(rows between 1 PRECEDING and current row) from t1
 {code}
 results in 
 {noformat}
 :1}}
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:271)
   ... 9 more
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:180)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage$GenericUDAFAverageEvaluatorDouble$1.getNextResult(GenericUDAFAverage.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStreamingEvaluator$SumAvgEnhancer.iterate(GenericUDAFStreamingEvaluator.java:166)
   at 
 org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:185)
   at 
 org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:348)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:318)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.processOp(PTFOperator.java:131)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:262)
   ... 9 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 24043: DefaultHiveAuthorizationProvider should not initialize a new HiveConf

2014-07-29 Thread Thejas Nair


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24043/#review48996
---



metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/24043/#comment85820

I think it would be safer to consider a value difference in case to be a 
different setting. Eg, some fields like METASTOREPWD,METASTOREDIRECTORY can be 
case sensitive.

I realize that equalsIgnoreCase is what old code was doing, but since we 
are anyway changing this function, I think it makes sense to fix it.




ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java
https://reviews.apache.org/r/24043/#comment85841

I think INFO or DEBUG level is better here. Also, printing changing it to 
some prefix in in the message would be useful. eg -

LOG.INFO(Current user + currentUserName + , Current Roles + 
currentRoles);




ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java
https://reviews.apache.org/r/24043/#comment85842

same comment as above




ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
https://reviews.apache.org/r/24043/#comment85844

why is this needed ?



- Thejas Nair


On July 29, 2014, 7:18 a.m., Navis Ryu wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24043/
 ---
 
 (Updated July 29, 2014, 7:18 a.m.)
 
 
 Review request for hive.
 
 
 Bugs: HIVE-6437
 https://issues.apache.org/jira/browse/HIVE-6437
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 During a HS2 connection, every SessionState got initializes a new 
 DefaultHiveAuthorizationProvider object (on stock configs).
 
 In turn, DefaultHiveAuthorizationProvider carries a {{new HiveConf(…)}} that 
 may prove too expensive, and unnecessary to do, since SessionState itself 
 sends in a fully applied HiveConf to it in the first place.
 
 
 Diffs
 -
 
   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ae7cd53 
   
 contrib/src/java/org/apache/hadoop/hive/contrib/metastore/hooks/TestURLHook.java
  39562ea 
   contrib/src/test/queries/clientnegative/url_hook.q c346432 
   contrib/src/test/queries/clientpositive/url_hook.q PRE-CREATION 
   contrib/src/test/results/clientnegative/url_hook.q.out 601fd93 
   contrib/src/test/results/clientpositive/url_hook.q.out PRE-CREATION 
   data/conf/hive-site.xml fe8080a 
   itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java 
 e8d405d 
   
 itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java
  0bb022e 
   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 2fefa06 
   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
 5cc1cd8 
   
 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
 d26183b 
   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
 5add436 
   metastore/src/java/org/apache/hadoop/hive/metastore/RawStoreProxy.java 
 1cf09d4 
   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 81323f6 
   
 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/DefaultHiveAuthorizationProvider.java
  2fa512c 
   
 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/StorageBasedAuthorizationProvider.java
  0dfd997 
   
 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveRoleGrant.java
  ce07f32 
   
 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java
  ce12edb 
   ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java d218271 
   ql/src/test/queries/clientnegative/authorization_cannot_create_all_role.q 
 de91e91 
   
 ql/src/test/queries/clientnegative/authorization_cannot_create_default_role.q 
 42a42f6 
   ql/src/test/queries/clientnegative/authorization_cannot_create_none_role.q 
 0d14cde 
   ql/src/test/queries/clientnegative/authorization_caseinsensitivity.q 
 d5ea284 
   ql/src/test/queries/clientnegative/authorization_drop_db_cascade.q edeae9b 
   ql/src/test/queries/clientnegative/authorization_drop_db_empty.q 46d4d0f 
   ql/src/test/queries/clientnegative/authorization_drop_role_no_admin.q 
 a7aa17f 
   ql/src/test/queries/clientnegative/authorization_priv_current_role_neg.q 
 463358a 
   ql/src/test/queries/clientnegative/authorization_role_cycles1.q a819d20 
   ql/src/test/queries/clientnegative/authorization_role_cycles2.q 423f030 
   ql/src/test/queries/clientnegative/authorization_role_grant.q c5c500a 
   ql/src/test/queries/clientnegative/authorization_role_grant2.q 7fdf157

[jira] [Commented] (HIVE-6437) DefaultHiveAuthorizationProvider should not initialize a new HiveConf

2014-07-29 Thread Thejas M Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078163#comment-14078163
 ] 

Thejas M Nair commented on HIVE-6437:
-

Added comments to review board.


 DefaultHiveAuthorizationProvider should not initialize a new HiveConf
 -

 Key: HIVE-6437
 URL: https://issues.apache.org/jira/browse/HIVE-6437
 Project: Hive
  Issue Type: Bug
  Components: Configuration
Affects Versions: 0.13.0
Reporter: Harsh J
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-6437.1.patch.txt, HIVE-6437.2.patch.txt, 
 HIVE-6437.3.patch.txt, HIVE-6437.4.patch.txt, HIVE-6437.5.patch.txt


 During a HS2 connection, every SessionState got initializes a new 
 DefaultHiveAuthorizationProvider object (on stock configs).
 In turn, DefaultHiveAuthorizationProvider carries a {{new HiveConf(…)}} that 
 may prove too expensive, and unnecessary to do, since SessionState itself 
 sends in a fully applied HiveConf to it in the first place.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7068) Integrate AccumuloStorageHandler

2014-07-29 Thread Josh Elser (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Elser updated HIVE-7068:
-

Attachment: HIVE-7068.2.patch

Minor updates to the patch:

* Removes unnecessary whitespace/javadoc
* Adds a better exception when Accumulo connection information isn't in the
hiveconf as required.
* Pulls in more upstream changes from trunk
* Fixes accumulo qtest after HIVE-5771

Also re-trigger HIVE QA which appear to have failed for other reasons on the
last patch. I'll update reviewboard as well if anyone wants to see the changes.

Integrate AccumuloStorageHandler

Key: HIVE-7068
URL: https://issues.apache.org/jira/browse/HIVE-7068
Project: Hive
Issue Type: New Feature
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 0.14.0

Attachments: HIVE-7068.1.patch, HIVE-7068.2.patch

[Accumulo|http://accumulo.apache.org] is a BigTable-clone which is similar to
HBase. Some [initial
work|https://github.com/bfemiano/accumulo-hive-storage-manager] has been done
to support querying an Accumulo table using Hive already. It is not a
complete solution as, most notably, the current implementation presently
lacks support for INSERTs.
I would like to polish up the AccumuloStorageHandler (presently based on
0.10), implement missing basic functionality and compare it to the
HBaseStorageHandler (to ensure that we follow the same general usage
patterns).
I've also been in communication with [~bfem] (the initial author) who
expressed interest in working on this again. I hope to coordinate efforts
with him.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 23914: Add AccumuloStorageHandler

2014-07-29 Thread Josh Elser


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23914/
---

(Updated July 29, 2014, 6:57 p.m.)


Review request for hive.


Changes
---

Minor updates to the original patch:

* Removes unnecessary whitespace/javadoc
* Adds a better exception when Accumulo connection information isn't in the 
hiveconf as required.
* Pulls in more upstream changes from trunk
* Fixes accumulo qtest after HIVE-5771


Bugs: HIVE-7068
https://issues.apache.org/jira/browse/HIVE-7068


Repository: hive-git


Description
---

Adds an AccumuloStorageHandler which allows Accumulo tables to back Hive 
tables. Predicate pushdown, table range elimination, UTF8 and binary 
serialization, and more.


Diffs (updated)
-

  accumulo-handler/pom.xml PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloConnectionParameters.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloHiveConstants.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloHiveRow.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloStorageHandler.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/LazyAccumuloMap.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/LazyAccumuloRow.java 
PRE-CREATION 
  accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/Utils.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnEncoding.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnMapper.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnMappingFactory.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveAccumuloColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveAccumuloMapColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveAccumuloRowIdColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveColumn.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/InvalidColumnMappingException.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloRecordReader.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloSplit.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloTableInputFormat.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloTableOutputFormat.java
 PRE-CREATION 
  accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/package-info.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloPredicateHandler.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloRangeGenerator.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/NoSuchCompareOpException.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/NoSuchPrimitiveComparisonException.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/PrimitiveComparisonFilter.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/PushdownTuple.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/CompareOp.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/DoubleCompare.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/Equal.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/GreaterThan.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/GreaterThanOrEqual.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/IntCompare.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/LessThan.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/LessThanOrEqual.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/Like.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/LongCompare.java
 PRE-CREATION

[jira] [Updated] (HIVE-7434) beeline should not always enclose the output by default in CSV/TSV mode

2014-07-29 Thread Szehon Ho (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Szehon Ho updated HIVE-7434:


Resolution: Duplicate
Status: Resolved  (was: Patch Available)

 beeline should not always enclose the output by default in CSV/TSV mode
 ---

 Key: HIVE-7434
 URL: https://issues.apache.org/jira/browse/HIVE-7434
 Project: Hive
  Issue Type: Bug
  Components: CLI
Reporter: ferdinand xu
Assignee: Ferdinand Xu
 Attachments: HIVE-7434.patch, HIVE-7434.patch


 When using beeline in CSV/TSV mode (via command !outputformat csv) , the 
 output is always enclosed in single quotes. This is however not the case for 
 Hive CLI, so we need to make this enclose optional.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7434) beeline should not always enclose the output by default in CSV/TSV mode

2014-07-29 Thread Szehon Ho (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-7434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078186#comment-14078186
 ] 

Szehon Ho commented on HIVE-7434:
-

Thanks, let's close this JIRA as duplicate and I'll look on that one then, 
otherwise this looks like its open for review :)

 beeline should not always enclose the output by default in CSV/TSV mode
 ---

 Key: HIVE-7434
 URL: https://issues.apache.org/jira/browse/HIVE-7434
 Project: Hive
  Issue Type: Bug
  Components: CLI
Reporter: ferdinand xu
Assignee: Ferdinand Xu
 Attachments: HIVE-7434.patch, HIVE-7434.patch


 When using beeline in CSV/TSV mode (via command !outputformat csv) , the 
 output is always enclosed in single quotes. This is however not the case for 
 Hive CLI, so we need to make this enclose optional.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7390) Make quote character optional and configurable in BeeLine CSV/TSV output

2014-07-29 Thread Szehon Ho (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078229#comment-14078229
 ] 

Szehon Ho commented on HIVE-7390:
-

Hi, I guess we will continue the discussion here from HIVE-7390.  First, thanks 
for incorporating my feedback from that partial patch.

So I read this patch, and it has one option:
{noformat} 'outputAsCLICSVFormat=[true/false]   display the output in the csv 
format as Hive command line\n \ {noformat}

The only difference is quote, right?   If my understanding is right, can't we 
have a new output format called 'quotedCSV', and change the default csv format 
to be unquoted, as was discussed earlier on this JIRA by Jim Halfpenny.  (He 
mentioned single-quote is not standard).  Some disadvantages of 
'outputAsCliCSVFormat' option is that its name is not very descriptive for 
users, and as Xuefu mentioned it is universal option even though it should only 
apply to CSV.  What do you think?

 Make quote character optional and configurable in BeeLine CSV/TSV output
 

 Key: HIVE-7390
 URL: https://issues.apache.org/jira/browse/HIVE-7390
 Project: Hive
  Issue Type: New Feature
  Components: Clients
Affects Versions: 0.13.1
Reporter: Jim Halfpenny
Assignee: Ferdinand Xu
 Attachments: HIVE-7390.1.patch, HIVE-7390.2.patch, HIVE-7390.3.patch, 
 HIVE-7390.patch


 Currently when either the CSV or TSV output formats are used in beeline each 
 column is wrapped in single quotes. Quote wrapping of columns should be 
 optional and the user should be able to choose the character used to wrap the 
 columns.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7488) pass column names being used for inputs to authorization api

2014-07-29 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-7488:


Attachment: HIVE-7488.6.patch

HIVE-7488.6.patch - fix NPE that caused tests to fail

 pass column names being used for inputs to authorization api
 

 Key: HIVE-7488
 URL: https://issues.apache.org/jira/browse/HIVE-7488
 Project: Hive
  Issue Type: Bug
  Components: Authorization
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Attachments: HIVE-7488.1.patch, HIVE-7488.2.patch, 
 HIVE-7488.3.patch.txt, HIVE-7488.4.patch, HIVE-7488.5.patch, HIVE-7488.6.patch


 HivePrivilegeObject in the authorization api has support for columns, but the 
 columns being used are not being populated for non grant-revoke queries.
 This is for enabling any implementation of the api to use this column 
 information for its authorization decisions.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7542) Cannot import text data to Hive tables with RCFile storage

2014-07-29 Thread akshay (JIRA)

akshay created HIVE-7542:


 Summary: Cannot import text data to Hive tables with RCFile storage
 Key: HIVE-7542
 URL: https://issues.apache.org/jira/browse/HIVE-7542
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Reporter: akshay
Priority: Critical






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7542) Cannot import text data to Hive tables with RCFile storage

2014-07-29 Thread akshay (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

akshay updated HIVE-7542:
-

Component/s: File Formats
 Compression

 Cannot import text data to Hive tables with RCFile storage
 --

 Key: HIVE-7542
 URL: https://issues.apache.org/jira/browse/HIVE-7542
 Project: Hive
  Issue Type: Bug
  Components: Compression, File Formats, HiveServer2
Reporter: akshay
Priority: Critical





--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7542) Cannot import text data to Hive tables with RCFile storage (DO NOT want to use intermediate tables)

2014-07-29 Thread akshay (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

akshay updated HIVE-7542:
-

Description: 
We plan to use RCFiles to create a data store as it can help store data in 
compressed format and the columnar format enables better querying for selective 
columns.

Problem: When we import data from text files (comma/tab delimited) into tables 
with RCFile storage format, we get an error as stated below:

Failed with exception Wrong file format. Please check the file's format.

FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.MoveTask

Workaround:
I know we can create an intermediate table.
Load data from text file to that table.
Then use insert into table rc_table select * from temp_text_file_table

But, we do not want to create intermediate tables as we have thousands of TB of 
data.
Summary: Cannot import text data to Hive tables with RCFile storage (DO 
NOT want to use intermediate tables)  (was: Cannot import text data to Hive 
tables with RCFile storage)

 Cannot import text data to Hive tables with RCFile storage (DO NOT want to 
 use intermediate tables)
 ---

 Key: HIVE-7542
 URL: https://issues.apache.org/jira/browse/HIVE-7542
 Project: Hive
  Issue Type: Bug
  Components: Compression, File Formats, HiveServer2
Reporter: akshay
Priority: Critical

 We plan to use RCFiles to create a data store as it can help store data in 
 compressed format and the columnar format enables better querying for 
 selective columns.
 Problem: When we import data from text files (comma/tab delimited) into 
 tables with RCFile storage format, we get an error as stated below:
 Failed with exception Wrong file format. Please check the file's format.
 FAILED: Execution Error, return code 1 from 
 org.apache.hadoop.hive.ql.exec.MoveTask
 Workaround:
 I know we can create an intermediate table.
 Load data from text file to that table.
 Then use insert into table rc_table select * from temp_text_file_table
 But, we do not want to create intermediate tables as we have thousands of TB 
 of data.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Why does SMB join generate hash table locally, even if input tables are large?

2014-07-29 Thread Pala M Muthaia

Hi,

I am testing SMB join for 2 large tables. The tables are bucketed and
sorted on the join column. I notice that even though the table is large,
Hive attempts to generate hash table for the 'small' table locally,
 similar to map join. Since the table is large in my case, the client runs
out of memory and the query fails.

I am using Hive 0.12 with the following settings:

set hive.optimize.bucketmapjoin=true;
set hive.optimize.bucketmapjoin.sortedmerge=true;
set hive.input.format =
org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;

My test query does a simple join and a select, no subqueries/nested queries
etc.

I understand why a (bucket) map join requires hash table generation, but
why is that included for an SMB join? Shouldn't a SMB join just spin up one
mapper for each bucket and perform a sort merge join directly on the mapper?


Thanks,
pala

[jira] [Assigned] (HIVE-7330) Create SparkTask


 [ 
https://issues.apache.org/jira/browse/HIVE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang reassigned HIVE-7330:
-

Assignee: Na Yang  (was: Chinna Rao Lalam)

 Create SparkTask
 

 Key: HIVE-7330
 URL: https://issues.apache.org/jira/browse/HIVE-7330
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Na Yang

 SparkTask handles the execution of SparkWork. It will execute a graph of map 
 and reduce work using a SparkClient instance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-7336) Create MapFunction


 [ 
https://issues.apache.org/jira/browse/HIVE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang resolved HIVE-7336.
---

   Resolution: Fixed
Fix Version/s: spark-branch

HiveMapFunction was introduced in HIVE-7370 with basic functionality in place. 
Improvement might be needed down the road, for which we will create specific 
JIRA for that. Closed this as fixed.

 Create MapFunction
 --

 Key: HIVE-7336
 URL: https://issues.apache.org/jira/browse/HIVE-7336
 Project: Hive
  Issue Type: Sub-task
Reporter: Xuefu Zhang
 Fix For: spark-branch


 Wrap Hive's map-side data processing for Spark.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (HIVE-7336) Create MapFunction


 [ 
https://issues.apache.org/jira/browse/HIVE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang reassigned HIVE-7336:
-

Assignee: Xuefu Zhang

 Create MapFunction
 --

 Key: HIVE-7336
 URL: https://issues.apache.org/jira/browse/HIVE-7336
 Project: Hive
  Issue Type: Sub-task
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: spark-branch


 Wrap Hive's map-side data processing for Spark.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7336) Create MapFunction


 [ 
https://issues.apache.org/jira/browse/HIVE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-7336:
--

Component/s: Spark

 Create MapFunction
 --

 Key: HIVE-7336
 URL: https://issues.apache.org/jira/browse/HIVE-7336
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: spark-branch


 Wrap Hive's map-side data processing for Spark.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-7337) Create ReduceFunction


 [ 
https://issues.apache.org/jira/browse/HIVE-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang resolved HIVE-7337.
---

   Resolution: Fixed
Fix Version/s: spark-branch
 Assignee: Xuefu Zhang

HiveReduceFunction was introduced in HIVE-7370 with basic functionality in 
place. Improvement might be needed down the road, for which we will create 
specific JIRA for that. Closed this as fixed.

 Create ReduceFunction
 -

 Key: HIVE-7337
 URL: https://issues.apache.org/jira/browse/HIVE-7337
 Project: Hive
  Issue Type: Sub-task
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: spark-branch


 Wrap Hive's reduce-side data processing for Spark.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (HIVE-7382) Create a MiniSparkCluster and set up a testing framework


 [ 
https://issues.apache.org/jira/browse/HIVE-7382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang reassigned HIVE-7382:
-

Assignee: Szehon Ho

 Create a MiniSparkCluster and set up a testing framework
 

 Key: HIVE-7382
 URL: https://issues.apache.org/jira/browse/HIVE-7382
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Szehon Ho

 To automatically test Hive functionality over Spark execution engine, we need 
 to create a test framework that can execute Hive queries with Spark as the 
 backend. For that, we should create a MiniSparkCluser for this, similar to 
 other execution engines.
 Spark has a way to create a local cluster with a few processes in the local 
 machine, each process is a work node. It's fairly close to a real Spark 
 cluster. Our mini cluster can be based on that.
 For more info, please refer to the design doc on wiki.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7068) Integrate AccumuloStorageHandler


[ 
https://issues.apache.org/jira/browse/HIVE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078503#comment-14078503
 ] 

Hive QA commented on HIVE-7068:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12658459/HIVE-7068.2.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5955 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/93/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/93/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-93/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12658459

 Integrate AccumuloStorageHandler
 

 Key: HIVE-7068
 URL: https://issues.apache.org/jira/browse/HIVE-7068
 Project: Hive
  Issue Type: New Feature
Reporter: Josh Elser
Assignee: Josh Elser
 Fix For: 0.14.0

 Attachments: HIVE-7068.1.patch, HIVE-7068.2.patch


 [Accumulo|http://accumulo.apache.org] is a BigTable-clone which is similar to 
 HBase. Some [initial 
 work|https://github.com/bfemiano/accumulo-hive-storage-manager] has been done 
 to support querying an Accumulo table using Hive already. It is not a 
 complete solution as, most notably, the current implementation presently 
 lacks support for INSERTs.
 I would like to polish up the AccumuloStorageHandler (presently based on 
 0.10), implement missing basic functionality and compare it to the 
 HBaseStorageHandler (to ensure that we follow the same general usage 
 patterns).
 I've also been in communication with [~bfem] (the initial author) who 
 expressed interest in working on this again. I hope to coordinate efforts 
 with him.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7535) Make use of number of nulls column statistics in filter rule

2014-07-29 Thread Gunther Hagleitner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-7535:
-

   Resolution: Fixed
Fix Version/s: (was: 0.13.0)
   0.14.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks [~prasanth_j]!

 Make use of number of nulls column statistics in filter rule
 

 Key: HIVE-7535
 URL: https://issues.apache.org/jira/browse/HIVE-7535
 Project: Hive
  Issue Type: Sub-task
  Components: Query Processor, Statistics
Affects Versions: 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
Priority: Minor
 Fix For: 0.14.0

 Attachments: HIVE-7535.1.patch, HIVE-7535.2.patch


 The filter rule does not make use of number of nulls column statistics for 
 IS NULL and IS NOT NULL expression evaluation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7543) Cleanup of org.apache.hive.service.auth package

Lars Francke created HIVE-7543:
--

 Summary: Cleanup of org.apache.hive.service.auth package
 Key: HIVE-7543
 URL: https://issues.apache.org/jira/browse/HIVE-7543
 Project: Hive
  Issue Type: Improvement
  Components: Authentication
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor


While trying to understand Hive's Thrift and Auth code I found some 
inconsistencies and complaints using Hive's own Checkstyle rules. My IDE and 
Sonar complained as well so I've taken the opportunity to clean this package up.

I'll follow up with a list of important changes tomorrow.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7068) Integrate AccumuloStorageHandler

2014-07-29 Thread Josh Elser (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Elser updated HIVE-7068:
-

Attachment: HIVE-7068.3.patch

Sorry, found another minor issue with serialization of strings as compared to
what HBaseStorageHandler does. New patch allows binary encoding to be specified
on strings without error (falls back to UTF8 serialization). Added a test for
it too, and cleaned up some other nits I saw in fixing the bug.

Integrate AccumuloStorageHandler

Key: HIVE-7068
URL: https://issues.apache.org/jira/browse/HIVE-7068
Project: Hive
Issue Type: New Feature
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 0.14.0

Attachments: HIVE-7068.1.patch, HIVE-7068.2.patch, HIVE-7068.3.patch

--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 24073: HIVE-7543: Cleanup of org.apache.hive.service.auth package


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24073/
---

Review request for hive.


Bugs: HIVE-7543
https://issues.apache.org/jira/browse/HIVE-7543


Repository: hive-git


Description
---

While trying to understand Hive's Thrift and Auth code I found some 
inconsistencies and complaints using Hive's own Checkstyle rules. My IDE and 
Sonar complained as well so I've taken the opportunity to clean this package up.


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ae7cd53 
  
itests/hive-unit/src/test/java/org/apache/hive/service/auth/TestCustomAuthentication.java
 ece54a8 
  
service/src/java/org/apache/hive/service/auth/AnonymousAuthenticationProviderImpl.java
 4db0022 
  
service/src/java/org/apache/hive/service/auth/AuthenticationProviderFactory.java
 e51d4f4 
  
service/src/java/org/apache/hive/service/auth/CustomAuthenticationProviderImpl.java
 7094b89 
  service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 72b3e7e 
  service/src/java/org/apache/hive/service/auth/HttpAuthUtils.java 91d7188 
  
service/src/java/org/apache/hive/service/auth/HttpAuthenticationException.java 
bf6be47 
  service/src/java/org/apache/hive/service/auth/HttpCLIServiceUGIProcessor.java 
8368938 
  service/src/java/org/apache/hive/service/auth/KerberosSaslHelper.java 4b70558 
  
service/src/java/org/apache/hive/service/auth/LdapAuthenticationProviderImpl.java
 5342214 
  
service/src/java/org/apache/hive/service/auth/PamAuthenticationProviderImpl.java
 5e48d13 
  
service/src/java/org/apache/hive/service/auth/PasswdAuthenticationProvider.java 
2d0da3a 
  
service/src/java/org/apache/hive/service/auth/PasswordAuthenticationProvider.java
 PRE-CREATION 
  service/src/java/org/apache/hive/service/auth/PlainSaslHelper.java dd788c6 
  service/src/java/org/apache/hive/service/auth/PlainSaslServer.java 79c44e6 
  service/src/java/org/apache/hive/service/auth/SaslQOP.java 0b2e7a2 
  service/src/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java 
3be4b4b 
  service/src/java/org/apache/hive/service/auth/TSubjectAssumingTransport.java 
d0468b3 
  service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java 
99ef8bc 

Diff: https://reviews.apache.org/r/24073/diff/


Testing
---


Thanks,

Lars Francke

[jira] [Commented] (HIVE-7543) Cleanup of org.apache.hive.service.auth package


[ 
https://issues.apache.org/jira/browse/HIVE-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078519#comment-14078519
 ] 

Lars Francke commented on HIVE-7543:


https://reviews.apache.org/r/24073/

 Cleanup of org.apache.hive.service.auth package
 ---

 Key: HIVE-7543
 URL: https://issues.apache.org/jira/browse/HIVE-7543
 Project: Hive
  Issue Type: Improvement
  Components: Authentication
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor

 While trying to understand Hive's Thrift and Auth code I found some 
 inconsistencies and complaints using Hive's own Checkstyle rules. My IDE and 
 Sonar complained as well so I've taken the opportunity to clean this package 
 up.
 I'll follow up with a list of important changes tomorrow.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 23914: Add AccumuloStorageHandler

2014-07-29 Thread Josh Elser


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23914/
---

(Updated July 29, 2014, 10:34 p.m.)


Review request for hive.


Changes
---

Fixes minor issue with serialization of strings as compared to what 
HBaseStorageHandler does. Allows binary encoding to be specified on strings 
without error (falls back to UTF8 serialization)


Bugs: HIVE-7068
https://issues.apache.org/jira/browse/HIVE-7068


Repository: hive-git


Description
---

Adds an AccumuloStorageHandler which allows Accumulo tables to back Hive 
tables. Predicate pushdown, table range elimination, UTF8 and binary 
serialization, and more.


Diffs (updated)
-

  accumulo-handler/pom.xml PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloConnectionParameters.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloHiveConstants.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloHiveRow.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloStorageHandler.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/LazyAccumuloMap.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/LazyAccumuloRow.java 
PRE-CREATION 
  accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/Utils.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnEncoding.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnMapper.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/ColumnMappingFactory.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveAccumuloColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveAccumuloMapColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveAccumuloRowIdColumnMapping.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/HiveColumn.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/columns/InvalidColumnMappingException.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloRecordReader.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloSplit.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloTableInputFormat.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloTableOutputFormat.java
 PRE-CREATION 
  accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/package-info.java 
PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloPredicateHandler.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloRangeGenerator.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/NoSuchCompareOpException.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/NoSuchPrimitiveComparisonException.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/PrimitiveComparisonFilter.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/PushdownTuple.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/CompareOp.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/DoubleCompare.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/Equal.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/GreaterThan.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/GreaterThanOrEqual.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/IntCompare.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/LessThan.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/LessThanOrEqual.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/Like.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/LongCompare.java
 PRE-CREATION 
  
accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/compare/NotEqual.java
 PRE-CREATION

[jira] [Commented] (HIVE-7543) Cleanup of org.apache.hive.service.auth package

2014-07-29 Thread Damien Carol (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078526#comment-14078526
 ] 

Damien Carol commented on HIVE-7543:


Could you add your patch file in the JIRA ticket ? thanks in advance.

 Cleanup of org.apache.hive.service.auth package
 ---

 Key: HIVE-7543
 URL: https://issues.apache.org/jira/browse/HIVE-7543
 Project: Hive
  Issue Type: Improvement
  Components: Authentication
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor

 While trying to understand Hive's Thrift and Auth code I found some 
 inconsistencies and complaints using Hive's own Checkstyle rules. My IDE and 
 Sonar complained as well so I've taken the opportunity to clean this package 
 up.
 I'll follow up with a list of important changes tomorrow.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7536) Make use of decimal column statistics in statistics annotation

2014-07-29 Thread Gunther Hagleitner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-7536:
-

   Resolution: Fixed
Fix Version/s: (was: 0.13.0)
   0.14.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks [~prasanth_j]!

 Make use of decimal column statistics in statistics annotation
 --

 Key: HIVE-7536
 URL: https://issues.apache.org/jira/browse/HIVE-7536
 Project: Hive
  Issue Type: Sub-task
  Components: Query Processor, Statistics
Affects Versions: 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
Priority: Minor
 Fix For: 0.14.0

 Attachments: HIVE-7536.1.patch


 HIVE-6701 added decimal column statistics. The statistics annotation 
 optimizer should make use of decimal column statistics as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-4329) HCatalog should use getHiveRecordWriter rather than getRecordWriter

2014-07-29 Thread David Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078537#comment-14078537
 ] 

David Chen commented on HIVE-4329:
--

I think I am close to getting this to work. Writing to Parquet seems to now be 
working for everything except for external tables. I am fairly certain that the 
reason why Avro is still not working is due to a table property for the Avro 
schema being missing.

 HCatalog should use getHiveRecordWriter rather than getRecordWriter
 ---

 Key: HIVE-4329
 URL: https://issues.apache.org/jira/browse/HIVE-4329
 Project: Hive
  Issue Type: Bug
  Components: HCatalog, Serializers/Deserializers
Affects Versions: 0.10.0
 Environment: discovered in Pig, but it looks like the root cause 
 impacts all non-Hive users
Reporter: Sean Busbey
Assignee: David Chen

 Attempting to write to a HCatalog defined table backed by the AvroSerde fails 
 with the following stacktrace:
 {code}
 java.lang.ClassCastException: org.apache.hadoop.io.NullWritable cannot be 
 cast to org.apache.hadoop.io.LongWritable
   at 
 org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat$1.write(AvroContainerOutputFormat.java:84)
   at 
 org.apache.hcatalog.mapreduce.FileRecordWriterContainer.write(FileRecordWriterContainer.java:253)
   at 
 org.apache.hcatalog.mapreduce.FileRecordWriterContainer.write(FileRecordWriterContainer.java:53)
   at 
 org.apache.hcatalog.pig.HCatBaseStorer.putNext(HCatBaseStorer.java:242)
   at org.apache.hcatalog.pig.HCatStorer.putNext(HCatStorer.java:52)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
   at 
 org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:559)
   at 
 org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85)
 {code}
 The proximal cause of this failure is that the AvroContainerOutputFormat's 
 signature mandates a LongWritable key and HCat's FileRecordWriterContainer 
 forces a NullWritable. I'm not sure of a general fix, other than redefining 
 HiveOutputFormat to mandate a WritableComparable.
 It looks like accepting WritableComparable is what's done in the other Hive 
 OutputFormats, and there's no reason AvroContainerOutputFormat couldn't also 
 be changed, since it's ignoring the key. That way fixing things so 
 FileRecordWriterContainer can always use NullWritable could get spun into a 
 different issue?
 The underlying cause for failure to write to AvroSerde tables is that 
 AvroContainerOutputFormat doesn't meaningfully implement getRecordWriter, so 
 fixing the above will just push the failure into the placeholder RecordWriter.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7543) Cleanup of org.apache.hive.service.auth package


 [ 
https://issues.apache.org/jira/browse/HIVE-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-7543:
---

Attachment: HIVE-7543.1.patch

Sure thing, here you go.

 Cleanup of org.apache.hive.service.auth package
 ---

 Key: HIVE-7543
 URL: https://issues.apache.org/jira/browse/HIVE-7543
 Project: Hive
  Issue Type: Improvement
  Components: Authentication
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-7543.1.patch


 While trying to understand Hive's Thrift and Auth code I found some 
 inconsistencies and complaints using Hive's own Checkstyle rules. My IDE and 
 Sonar complained as well so I've taken the opportunity to clean this package 
 up.
 I'll follow up with a list of important changes tomorrow.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 24075: HIVE-6123: Implement checkstyle in maven


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24075/#review49041
---



checkstyle/checkstyle.xml
https://reviews.apache.org/r/24075/#comment85877

This is not supported in later Checkstyle versions anymore because the 
underlying issue has been fixed with Java 5 and its well defined memory model.


- Lars Francke


On July 29, 2014, 11:02 p.m., Lars Francke wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24075/
 ---
 
 (Updated July 29, 2014, 11:02 p.m.)
 
 
 Review request for hive.
 
 
 Bugs: HIVE-6123
 https://issues.apache.org/jira/browse/HIVE-6123
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 This patch makes mvn checkstyle:checkstyle work on my machine. Builds on the 
 previous patch by Remus, thank you.
 
 
 Diffs
 -
 
   checkstyle/checkstyle.xml bdd8f58 
   pom.xml b5a5697 
 
 Diff: https://reviews.apache.org/r/24075/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Lars Francke

Review Request 24075: HIVE-6123: Implement checkstyle in maven


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24075/
---

Review request for hive.


Bugs: HIVE-6123
https://issues.apache.org/jira/browse/HIVE-6123


Repository: hive-git


Description
---

This patch makes mvn checkstyle:checkstyle work on my machine. Builds on the 
previous patch by Remus, thank you.


Diffs
-

  checkstyle/checkstyle.xml bdd8f58 
  pom.xml b5a5697 

Diff: https://reviews.apache.org/r/24075/diff/


Testing
---


Thanks,

Lars Francke

[jira] [Updated] (HIVE-6123) Implement checkstyle in maven


 [ 
https://issues.apache.org/jira/browse/HIVE-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated HIVE-6123:
---

Attachment: HIVE-6123.2.patch

 Implement checkstyle in maven
 -

 Key: HIVE-6123
 URL: https://issues.apache.org/jira/browse/HIVE-6123
 Project: Hive
  Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Lars Francke
 Attachments: HIVE-6123.1.patch, HIVE-6123.2.patch


 ant had a checkstyle target, we should do something similar for maven



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (HIVE-6123) Implement checkstyle in maven


 [ 
https://issues.apache.org/jira/browse/HIVE-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke reassigned HIVE-6123:
--

Assignee: Lars Francke

 Implement checkstyle in maven
 -

 Key: HIVE-6123
 URL: https://issues.apache.org/jira/browse/HIVE-6123
 Project: Hive
  Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Lars Francke
 Attachments: HIVE-6123.1.patch, HIVE-6123.2.patch


 ant had a checkstyle target, we should do something similar for maven



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6123) Implement checkstyle in maven


[ 
https://issues.apache.org/jira/browse/HIVE-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078559#comment-14078559
 ] 

Lars Francke commented on HIVE-6123:


I've taken the liberty to work on this: https://reviews.apache.org/r/24075/

 Implement checkstyle in maven
 -

 Key: HIVE-6123
 URL: https://issues.apache.org/jira/browse/HIVE-6123
 Project: Hive
  Issue Type: Sub-task
Reporter: Brock Noland
 Attachments: HIVE-6123.1.patch, HIVE-6123.2.patch


 ant had a checkstyle target, we should do something similar for maven



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 24076: HIVE-7107: Clean up HiveDrivers and fix an issue


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24076/
---

Review request for hive.


Bugs: HIVE-7107
https://issues.apache.org/jira/browse/HIVE-7107


Repository: hive-git


Description
---

The HiveServer1 driver does not adhere to the JDBC standard of returning null 
when it can't handle a connection URL.

I realize that HS1 is scheduled to be removed but I submitted this before 
0.13.1 was released and maybe 0.13.2 will be released still. Also I think 
there's a chance that some distributions will keep HS1 in for a bit longer and 
this patch fixes an issue with it.


Diffs
-

  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java 
e2d9b42 
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveDriver.java 68c0788 
  jdbc/src/java/org/apache/hive/jdbc/HiveDriver.java 6e248d6 

Diff: https://reviews.apache.org/r/24076/diff/


Testing
---


Thanks,

Lars Francke

[jira] [Commented] (HIVE-7107) Clean up HiveDrivers and fix an issue


[ 
https://issues.apache.org/jira/browse/HIVE-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078567#comment-14078567
 ] 

Lars Francke commented on HIVE-7107:


https://reviews.apache.org/r/24076/

I realize that HS1 is scheduled to be removed but I submitted this before 
0.13.1 was released and maybe 0.13.2 will be released still. Also I think 
there's a chance that some distributions will keep HS1 in for a bit longer and 
this patch fixes an issue with it. I also don't see any harm in maintaining 
code that's scheduled to be removed at some point.

 Clean up HiveDrivers and fix an issue
 -

 Key: HIVE-7107
 URL: https://issues.apache.org/jira/browse/HIVE-7107
 Project: Hive
  Issue Type: Bug
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
 Attachments: HIVE-7107.2.patch


 The HiveServer1 driver does not adhere to the JDBC standard of returning null 
 when it can't handle a connection URL. It instead throws an exception which 
 leads to subsequent exceptions (from other drivers, i.e. the HiveServer2 one) 
 being swallowed.
 This is what you'd see:
 {quote}
 Error: Invalid URL: 
 jdbc:hive2://localhost:1/default;principal=hive/_h...@example.com 
 (state=08S01,code=0)
 java.sql.SQLException: Invalid URL: 
 jdbc:hive2://localhost:1/default;principal=hive/_h...@example.com
   at 
 org.apache.hadoop.hive.jdbc.HiveConnection.init(HiveConnection.java:86)
 {quote}
 In addition this patch cleans up the drivers a bit.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-4329) HCatalog should use getHiveRecordWriter rather than getRecordWriter

2014-07-29 Thread David Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078586#comment-14078586
 ] 

David Chen commented on HIVE-4329:
--

Correction: Parquet is working for everything except tables with static 
partitioning. I am pretty sure the root cause has to do with missing table 
properties.

 HCatalog should use getHiveRecordWriter rather than getRecordWriter
 ---

 Key: HIVE-4329
 URL: https://issues.apache.org/jira/browse/HIVE-4329
 Project: Hive
  Issue Type: Bug
  Components: HCatalog, Serializers/Deserializers
Affects Versions: 0.10.0
 Environment: discovered in Pig, but it looks like the root cause 
 impacts all non-Hive users
Reporter: Sean Busbey
Assignee: David Chen

 Attempting to write to a HCatalog defined table backed by the AvroSerde fails 
 with the following stacktrace:
 {code}
 java.lang.ClassCastException: org.apache.hadoop.io.NullWritable cannot be 
 cast to org.apache.hadoop.io.LongWritable
   at 
 org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat$1.write(AvroContainerOutputFormat.java:84)
   at 
 org.apache.hcatalog.mapreduce.FileRecordWriterContainer.write(FileRecordWriterContainer.java:253)
   at 
 org.apache.hcatalog.mapreduce.FileRecordWriterContainer.write(FileRecordWriterContainer.java:53)
   at 
 org.apache.hcatalog.pig.HCatBaseStorer.putNext(HCatBaseStorer.java:242)
   at org.apache.hcatalog.pig.HCatStorer.putNext(HCatStorer.java:52)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
   at 
 org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:559)
   at 
 org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85)
 {code}
 The proximal cause of this failure is that the AvroContainerOutputFormat's 
 signature mandates a LongWritable key and HCat's FileRecordWriterContainer 
 forces a NullWritable. I'm not sure of a general fix, other than redefining 
 HiveOutputFormat to mandate a WritableComparable.
 It looks like accepting WritableComparable is what's done in the other Hive 
 OutputFormats, and there's no reason AvroContainerOutputFormat couldn't also 
 be changed, since it's ignoring the key. That way fixing things so 
 FileRecordWriterContainer can always use NullWritable could get spun into a 
 different issue?
 The underlying cause for failure to write to AvroSerde tables is that 
 AvroContainerOutputFormat doesn't meaningfully implement getRecordWriter, so 
 fixing the above will just push the failure into the placeholder RecordWriter.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (HIVE-1960) Database feature should be documented


 [ 
https://issues.apache.org/jira/browse/HIVE-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke reassigned HIVE-1960:
--

Assignee: Lars Francke  (was: Carl Steinbach)

 Database feature should be documented
 -

 Key: HIVE-1960
 URL: https://issues.apache.org/jira/browse/HIVE-1960
 Project: Hive
  Issue Type: Improvement
  Components: Documentation
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor

 The database feature introduced by HIVE-675 should be documented somewhere.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-1960) Database feature should be documented


 [ 
https://issues.apache.org/jira/browse/HIVE-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke resolved HIVE-1960.


Resolution: Fixed

This has been documented on the Wiki by Lefty, me and others.

 Database feature should be documented
 -

 Key: HIVE-1960
 URL: https://issues.apache.org/jira/browse/HIVE-1960
 Project: Hive
  Issue Type: Improvement
  Components: Documentation
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor

 The database feature introduced by HIVE-675 should be documented somewhere.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7480) Support Windowing Functions

2014-07-29 Thread Laljo John Pullokkaran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7480:
-

Attachment: HIVE-7480.patch

 Support Windowing Functions
 ---

 Key: HIVE-7480
 URL: https://issues.apache.org/jira/browse/HIVE-7480
 Project: Hive
  Issue Type: Sub-task
Reporter: Laljo John Pullokkaran
Assignee: Laljo John Pullokkaran
 Attachments: HIVE-7480.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7480) Support Windowing Functions

2014-07-29 Thread Laljo John Pullokkaran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-7480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7480:
-

Status: Patch Available  (was: Open)

 Support Windowing Functions
 ---

 Key: HIVE-7480
 URL: https://issues.apache.org/jira/browse/HIVE-7480
 Project: Hive
  Issue Type: Sub-task
Reporter: Laljo John Pullokkaran
Assignee: Laljo John Pullokkaran
 Attachments: HIVE-7480.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7544) Changes related to TEZ-1288 (FastTezSerialization)

2014-07-29 Thread Rajesh Balamohan (JIRA)

Rajesh Balamohan created HIVE-7544:
--

 Summary: Changes related to TEZ-1288 (FastTezSerialization)
 Key: HIVE-7544
 URL: https://issues.apache.org/jira/browse/HIVE-7544
 Project: Hive
  Issue Type: Sub-task
Reporter: Rajesh Balamohan
Assignee: Rajesh Balamohan


Add ability to make use of TezBytesWritableSerialization.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-4933) Can't use alias directly before OVER clause