date:20131216


[ 
https://issues.apache.org/jira/browse/HIVE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848995#comment-13848995
 ] 

Hive QA commented on HIVE-2818:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618858/HIVE-2818.2.patch.txt

{color:red}ERROR:{color} -1 due to 195 failed/errored test(s), 4774 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_multiple
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_char1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_char2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition_authorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_varchar1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_varchar2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_limit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_excludeHadoop20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_reordering_values
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_partitioned
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binarysortable_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_join1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_nested_types
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_serde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_udf1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_union1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view_translate
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_char
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_date
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_varchar
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_join1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_serde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_join

[jira] [Commented] (HIVE-6037) Synchronize HiveConf with hive-default.xml.template and support show conf


[ 
https://issues.apache.org/jira/browse/HIVE-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849000#comment-13849000
 ] 

Hive QA commented on HIVE-6037:
---



{color:red}Overall{color}: -1 no tests executed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618865/HIVE-6037.1.patch.txt

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/650/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/650/console

Messages:
{noformat}
 This message was trimmed, see log for full details 
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/tmp
[mkdir] Created dir: 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/warehouse
[mkdir] Created dir: 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/tmp/conf
 [copy] Copying 4 files to 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/tmp/conf
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
hive-hcatalog-it-unit ---
[INFO] Compiling 7 source files to 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/test-classes
[WARNING] Note: Some input files use or override a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ 
hive-hcatalog-it-unit ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-it-unit ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.13.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hcatalog-it-unit ---
[INFO] Building jar: 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.13.0-SNAPSHOT-tests.jar
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ 
hive-hcatalog-it-unit ---
[INFO] Installing 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.13.0-SNAPSHOT.jar
 to 
/data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.13.0-SNAPSHOT/hive-hcatalog-it-unit-0.13.0-SNAPSHOT.jar
[INFO] Installing 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/pom.xml 
to 
/data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.13.0-SNAPSHOT/hive-hcatalog-it-unit-0.13.0-SNAPSHOT.pom
[INFO] Installing 
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.13.0-SNAPSHOT-tests.jar
 to 
/data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.13.0-SNAPSHOT/hive-hcatalog-it-unit-0.13.0-SNAPSHOT-tests.jar
[INFO] 
[INFO] 
[INFO] Building Hive Integration - Testing Utilities 0.13.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-util ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/util 
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
hive-it-util ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-util ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-util 
---
[INFO] Compiling 41 source files to 
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/classes
[WARNING] Note: Some input files use or override a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
hive-it-util ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/test/resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-util ---
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp
[mkdir] Created dir:

[jira] [Commented] (HIVE-5879) Fix spelling errors in hive-default.xml

2013-12-16 Thread Lefty Leverenz (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849013#comment-13849013
 ] 

Lefty Leverenz commented on HIVE-5879:
--

bq. Beside, I've made a patch pulling description and default values from 
hive-default.xml.template into HiveConf.ConfVars

Sweet!

The [Configuration 
Properties|https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties]
 wikidoc also has release information for some properties.  A handful have 
Added in: and two have Removed in: plus one with a behavior change.  Should 
this release information be merged with the descriptions?

TL;DR Here's the list: 

* mapred.reduce.tasks   Added In: 0.1
* hive.metastore.schema.verification   Added In: Hive 0.12.0 with HIVE-3764
* hive.security.command.whitelist   Added In: Hive 0.13.0 with HIVE-5400
* hive.security.metastore.authorization.manager   Added In: Hive 0.10.0
* hive.security.metastore.authenticator.manager   Added In: Hive 0.10.0
* hive.task.progress   Removed in: Hive 0.13.0 with HIVE-4518
* hive.metastore.local   Removed in: Hive 0.10 with HIVE-2585
* datanucleus.autoCreateSchema   In Hive 0.12.0 and later releases, 
datanucleus.autoCreateSchema is disabled if hive.metastore.schema.verification 
is true.
* _not in wiki, patch not committed yet_  hive.exec.parallel  added in 0.5.0 
(HIVE-549); default → true in 0.13.0 (HIVE-1033)

 Fix spelling errors in hive-default.xml
 ---

 Key: HIVE-5879
 URL: https://issues.apache.org/jira/browse/HIVE-5879
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.12.0
Reporter: Brock Noland
Assignee: Lefty Leverenz
Priority: Trivial
  Labels: documentation
 Fix For: 0.13.0

 Attachments: HIVE-5879.2.patch.txt, HIVE-5879.patch


 See 
 https://issues.apache.org/jira/browse/HIVE-5400?focusedCommentId=13830626page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13830626



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5975) [WebHCat] templeton mapreduce job failed if provide define parameters


[ 
https://issues.apache.org/jira/browse/HIVE-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849044#comment-13849044
 ] 

Hive QA commented on HIVE-5975:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618157/hive-5975.2.patch

{color:green}SUCCESS:{color} +1 4785 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/651/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/651/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618157

 [WebHCat] templeton mapreduce job failed if provide define parameters
 ---

 Key: HIVE-5975
 URL: https://issues.apache.org/jira/browse/HIVE-5975
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 0.12.0, 0.13.0
Reporter: shanyu zhao
Assignee: shanyu zhao
 Attachments: hive-5975.2.patch, hive-5975.patch


 Trying to submit a mapreduce job through templeton failed:
 curl -k -u user:pass -d user.name=user -d define=JobName=MRPiJob -d class=pi 
 -d arg=16 -d arg=100 -d jar=hadoop-mapreduce-examples.jar 
 https://xxx/templeton/v1/mapreduce/jar
 The error message is:
 Usage: org.apache.hadoop.examples.QuasiMonteCarlo nMaps nSamples
  Generic options supported are
  -conf configuration file specify an application configuration file
  -D property=value use value for given property
  -fs local|namenode:port specify a namenode
  -jt local|jobtracker:port specify a job tracker
  -files comma separated list of files specify comma separated files to be 
 copied to the map reduce cluster
  -libjars comma separated list of jars specify comma separated jar files to 
 include in the classpath.
  -archives comma separated list of archives specify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
  bin/hadoop command [genericOptions] [commandOptions]
 templeton: job failed with exit code 2
 Note that if we remove the define parameter it works fine.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: doc on predicate pushdown in joins

2013-12-16 Thread Lefty Leverenz


 This is old code, hard to attribute to  jiras


Okay, HIVE-2337 was way back in release 0.8.0 so I think we can safely omit
version information.  If anyone objects, I can link the wikidoc to that
ticket.  (And I've marked HIVE-2337 as being related to HIVE-279 in the
JIRA.)

Thanks for all the help.

-- Lefty


On Wed, Dec 11, 2013 at 7:03 PM, Harish Butani hbut...@hortonworks.comwrote:

 I see.  Let's leave it in.

 This is old code, hard to attribute to  jiras:
 - The PPD code comes from: HIVE-279, HIVE-2337
 - I cannot tell when the join condition parsing code was added.

 regards,
 Harish.

 On Dec 11, 2013, at 5:17 PM, Lefty Leverenz leftylever...@gmail.com
 wrote:

  Maybe we should remove the section on Hive Implementation here.
  It is in the Design doc; this information only concerns developers.

 But this is the Design doc (unless there's another one somewhere -- maybe
 attached to a JIRA ticket?) and it's in the Resources for Contributors part
 of the wiki, so it seems appropriate to me.  I'll delete the implementation
 section if that's your preference.

 Here are the links again, with fixes:

- Design Docshttps://cwiki.apache.org/confluence/display/Hive/DesignDocs 
 (bottom
of list)
- Predicate Pushdown 
 Ruleshttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-PredicatePushdownRules


 Speaking of JIRA tickets, is there one for this and should I add any
 version information?

 -- Lefty


 On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani hbut...@hortonworks.comwrote:

 getQualifiedAliases is a private method in JoinPPD.

 Maybe we should remove the section on Hive Implementation here. It is in
 the Design doc; this information only concerns developers.

 regards,
 Harish.


 On Dec 11, 2013, at 3:05 AM, Lefty Leverenz leftylever...@gmail.com
 wrote:

 Happy to fix the sentence and the link.  I pointed out the name change
 just so you would review it, so please don't apologize!

 One more question:  why am I not finding getQualifiedAliases() in the
 SemanticAnalyzer class?  It turns up in OpProcFactory.java with javadoc
 comments, but I can't find it anywhere in the API docs -- not even in the
 index (Hive 0.12.0 API http://hive.apache.org/docs/r0.12.0/api/):

 *getQMap()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html#getQMap()
  -
 Method in class 
 org.apache.hadoop.hive.ql.QTestUtilhttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html
  
 *getQualifiedName()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html#getQualifiedName()
  -
 Method in class 
 org.apache.hadoop.hive.serde2.typeinfo.TypeInfohttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html
  String
 representing the qualified type 
 name.*getQualifiers()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiers()
  -
 Method in class 
 org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html
  
 *getQualifiersSize()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiersSize()
  -
 Method in class 
 org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html
 Most mysterious.

 -- Lefty


 On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani 
 hbut...@hortonworks.comwrote:

 I can see why you would rename.

 But this sentence is not correct:
 'Hive enforces the predicate pushdown rules by these methods in the
 SemanticAnalyzer and JoinPPD classes:'

 It should be:
 Hive enforces the rules by these methods in the SemanticAnalyzer and
 JoinPPD classes:

 (The implementation involves both predicate pushdown and analyzing join
 conditions)
 Sorry about this.

 So the link should say 'Hive Outer Join Behavior'

 regards,
 Harish.


 On Dec 10, 2013, at 2:01 PM, Lefty Leverenz leftylever...@gmail.com
 wrote:

 How's this?  Hive 
 Implementationhttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-HiveImplementation

 Also, I moved the link on the Design Docs 
 pagehttps://cwiki.apache.org/confluence/display/Hive/DesignDocsfrom
 *Proposed* to *Other*.  (It's called SQL Outer Join Predicate Pushdown
 Ruleshttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior 
 which
 doesn't match the title, but seems okay because it's more descriptive.)

 -- Lefty


 On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani 
 hbut...@hortonworks.comwrote:

 You are correct, it is plural.

 regards,
 Harish.

 On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com
 wrote:

 Okay, then monospace with () after the method name is a good way to
 show them:  parseJoinCondition() and getQualifiedAlias() ... but I
 only found the latter pluralized, instead of

[jira] [Commented] (HIVE-3286) Explicit skew join on user provided condition


[ 
https://issues.apache.org/jira/browse/HIVE-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849088#comment-13849088
 ] 

Hive QA commented on HIVE-3286:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618867/HIVE-3286.16.patch.txt

{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 4790 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin_explicit
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_skewjoin_explicit_invalid1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_skewjoin_explicit_invalid2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_skewjoin_explicit_invalid3
{noformat}

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/652/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/652/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618867

 Explicit skew join on user provided condition
 -

 Key: HIVE-3286
 URL: https://issues.apache.org/jira/browse/HIVE-3286
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
 Attachments: D4287.11.patch, HIVE-3286.12.patch.txt, 
 HIVE-3286.13.patch.txt, HIVE-3286.14.patch.txt, HIVE-3286.15.patch.txt, 
 HIVE-3286.16.patch.txt, HIVE-3286.D4287.10.patch, HIVE-3286.D4287.5.patch, 
 HIVE-3286.D4287.6.patch, HIVE-3286.D4287.7.patch, HIVE-3286.D4287.8.patch, 
 HIVE-3286.D4287.9.patch


 Join operation on table with skewed data takes most of execution time 
 handling the skewed keys. But mostly we already know about that and even know 
 what is look like the skewed keys.
 If we can explicitly assign reducer slots for the skewed keys, total 
 execution time could be greatly shortened.
 As for a start, I've extended join grammar something like this.
 {code}
 select * from src a join src b on a.key=b.key skew on (a.key+1  50, a.key+1 
  100, a.key  150);
 {code}
 which means if above query is executed by 20 reducers, one reducer for 
 a.key+1  50, one reducer for 50 = a.key+1  100, one reducer for 99 = 
 a.key  150, and 17 reducers for others (could be extended to assign more 
 than one reducer later)
 This can be only used with common-inner-equi joins. And skew condition should 
 be composed of join keys only.
 Work till done now will be updated shortly after code cleanup.
 
 Skew expressions* in SKEW ON (expr, expr, ...) are evaluated sequentially 
 at runtime, and first 'true' one decides skew group for the row. Each skew 
 group has reserved partition slot(s), to which all rows in a group would be 
 assigned. 
 The number of partition slot reserved for each group is decided also at 
 runtime by simple calculation of percentage. If a skew group is CLUSTER BY 
 20 PERCENT and total partition slot (=number of reducer) is 20, that group 
 will reserve 4 partition slots, etc.
 DISTRIBUTE BY decides how the rows in a group is dispersed in the range of 
 reserved slots (If there is only one slot for a group, this is meaningless). 
 Currently, three distribution policies are available: RANDOM, KEYS, 
 expression. 
 1. RANDOM : rows of driver** alias are dispersed by random and rows of 
 non-driver alias are duplicated for all the slots (default if not specified)
 2. KEYS : determined by hash value of keys (same with previous)
 3. expression : determined by hash of object evaluated by user-provided 
 expression
 Only possible with inner, equi, common-joins. Not yet supports join tree 
 merging.
 Might be used by other RS users like SORT BY or GROUP BY
 If there exists column statistics for the key, it could be possible to apply 
 automatically.
 For example, if 20 reducers are used for the query below,
 {code}
 select count(*) from src a join src b on a.key=b.key skew on (
a.key = '0' CLUSTER BY 10 PERCENT,
b.key  '100' CLUSTER BY 20 PERCENT DISTRIBUTE BY upper(b.key),
cast(a.key as int)  300 CLUSTER BY 40 PERCENT DISTRIBUTE BY KEYS);
 {code}
 group-0 will reserve slots 6~7, group-1 8~11, group-2 12~19 and others will 
 reserve slots 0~5.
 For a row with key='0' from alias a, the row is randomly assigned in the 
 range of 6~7 (driver alias) : 6 or 7
 For a row with key='0' from alias b, the row is disributed for all slots in 
 6~7 (non-driver alias) : 6 and 7
 For a row with

[jira] [Updated] (HIVE-6022) Load statements with incorrect order of partitions put input files to unreadable places


 [ 
https://issues.apache.org/jira/browse/HIVE-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6022:
--

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Patch committed to trunk. Thanks to Teruyoshi for the contribution.

 Load statements with incorrect order of partitions put input files to 
 unreadable places
 ---

 Key: HIVE-6022
 URL: https://issues.apache.org/jira/browse/HIVE-6022
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Teruyoshi Zenmyo
Assignee: Teruyoshi Zenmyo
 Fix For: 0.13.0

 Attachments: HIVE-6022.1.patch, HIVE-6022.1.patch.txt, 
 HIVE-6022.2.patch.txt


 Load statements with incorrect order of partitions put input files to 
 incorrect paths. 
 {code}
 CREATE TABLE test_parts (c1 string, c2 int) PARTITIONED BY (p1 string,p2 
 string);
 LOAD DATA LOCAL INPATH '/opt/hive/examples/files/kv1.txt' OVERWRITE INTO 
 TABLE test_parts PARTITION (p2='p1', p1='p2')
 {code}
 The input file is located as below and the data is not readable.
 {code}
 % find /user/hive/warehouse/test_parts/
 /user/hive/warehouse/test_parts/
 /user/hive/warehouse/test_parts//p1=p2
 /user/hive/warehouse/test_parts//p1=p2/p2=p1
 /user/hive/warehouse/test_parts//p2=p1
 /user/hive/warehouse/test_parts//p2=p1/p1=p2
 /user/hive/warehouse/test_parts//p2=p1/p1=p2/.kv1.txt.crc
 /user/hive/warehouse/test_parts//p2=p1/p1=p2/kv1.txt
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Created] (HIVE-6038) Fix Tez branch to properly compile against hadoop-1 profile

2013-12-16 Thread Gunther Hagleitner (JIRA)

Gunther Hagleitner created HIVE-6038:


 Summary: Fix Tez branch to properly compile against hadoop-1 
profile
 Key: HIVE-6038
 URL: https://issues.apache.org/jira/browse/HIVE-6038
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
 Fix For: tez-branch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6038) Fix Tez branch to properly compile against hadoop-1 profile

2013-12-16 Thread Gunther Hagleitner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-6038:
-

Attachment: HIVE-6038.1.patch

 Fix Tez branch to properly compile against hadoop-1 profile
 ---

 Key: HIVE-6038
 URL: https://issues.apache.org/jira/browse/HIVE-6038
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
 Fix For: tez-branch

 Attachments: HIVE-6038.1.patch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Resolved] (HIVE-6038) Fix Tez branch to properly compile against hadoop-1 profile

2013-12-16 Thread Gunther Hagleitner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner resolved HIVE-6038.
--

Resolution: Fixed

Committed to branch.

 Fix Tez branch to properly compile against hadoop-1 profile
 ---

 Key: HIVE-6038
 URL: https://issues.apache.org/jira/browse/HIVE-6038
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
 Fix For: tez-branch

 Attachments: HIVE-6038.1.patch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Hive-trunk-hadoop2 - Build # 602 - Still Failing

Changes for Build #570
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #571
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #572
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #573
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #574
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #575
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #576

Changes for Build #577

Changes for Build #578

Changes for Build #579
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #580
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #581
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #582
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #583
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #584
[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)

[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #585
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)


Changes for Build #586
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #587
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore direct sql doesn't work with oracle (Sergey

Hive-trunk-h0.21 - Build # 2503 - Still Failing

Changes for Build #2471
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #2472
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #2473
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #2474
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #2475
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #2476
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #2477

Changes for Build #2478

Changes for Build #2479

Changes for Build #2480
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #2481

Changes for Build #2482
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #2483
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #2484
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #2485
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #2486
[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #2487
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)

[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)


Changes for Build #2488
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #2489
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore

Hive-trunk-hadoop2 - Build # 603 - Still Failing

Changes for Build #570
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #571
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #572
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #573
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #574
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #575
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #576

Changes for Build #577

Changes for Build #578

Changes for Build #579
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #580
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #581
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #582
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #583
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #584
[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)

[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #585
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)


Changes for Build #586
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #587
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore direct sql doesn't work with oracle (Sergey

Hive-trunk-h0.21 - Build # 2504 - Still Failing

Changes for Build #2471
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #2472
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #2473
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #2474
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #2475
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #2476
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #2477

Changes for Build #2478

Changes for Build #2479

Changes for Build #2480
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #2481

Changes for Build #2482
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #2483
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #2484
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #2485
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #2486
[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #2487
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)

[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)


Changes for Build #2488
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #2489
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore

Re: adding ANSI flag for hive

2013-12-16 Thread Alan Gates

A couple of thoughts on this:

1) If we did this I think we should have one flag, not many.  As Thejas points 
out, your test matrix goes insane when you have too many flags and hence things 
don't get properly tested.

2) We could do this in an incremental way, where we create this new ANSI flag 
and are clear with users that for a while this will be evolving.  That is, as 
we find new issues with data types, semantics, whatever, we will continue to 
change the behavior of this flag.  At some point in the future (as Thejas 
suggests, at a 1.0 release) we could make this the default behavior.  This 
avoids having to do a full sweep now and find everything that we want to change 
and make ANSI compliant and living with whatever we miss.

Alan.

On Dec 11, 2013, at 5:14 PM, Thejas Nair wrote:

 Having too many configs complicates things for the user, and also
 complicates the code, and you also end up having many untested
 combinations of config flags.
 I think we should identify a bunch of non compatible changes that we
 think are important, fix it in a branch and make a major version
 release (say 1.x).
 
 This is also related to HIVE-5875, where there is a discussion on
 switching the defaults for some of the configs to more desirable
 values, but non backward compatible values.
 
 On Wed, Dec 11, 2013 at 4:33 PM, Sergey Shelukhin
 ser...@hortonworks.com wrote:
 Hi.
 
 There's recently been some discussion about data type changes in Hive
 (double to decimal), and result changes for special cases like division by
 zero, etc., to bring it in compliance with MySQL (that's what JIRAs use an
 example; I am assuming ANSI SQL is meant).
 The latter are non-controversial (I guess), but for the former, performance
 may suffer and/or backward compat may be broken if Hive is brought in
 compliance.
 If fuller ANSI compat is sought in the future, there may be some even
 hairier issues such as double-quoted identifiers.
 
 In light of that, and also following MySQL, I wonder if we should add a
 flag, or set of flags, to HIVE to be able to force ANSI compliance.
 When this/ese flag/s is/are not set, for example, int/int division could
 return double for backward compat/perf, vectorization can skip the special
 case handling for division by zero/etc., etc.
 Wdyt?
 
 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender immediately
 and delete it from your system. Thank You.
 
 -- 
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to 
 which it is addressed and may contain information that is confidential, 
 privileged and exempt from disclosure under applicable law. If the reader 
 of this message is not the intended recipient, you are hereby notified that 
 any printing, copying, dissemination, distribution, disclosure or 
 forwarding of this communication is strictly prohibited. If you have 
 received this communication in error, please contact the sender immediately 
 and delete it from your system. Thank You.


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Review Request 16269: HIVE-6034 vectorized % doesn't handle zeroes the same way as non-vectorized


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16269/#review30454
---



ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NullUtil.java
https://reviews.apache.org/r/16269/#comment58256

Instead of NaN say the null data value. NaN is only for double.



ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NullUtil.java
https://reviews.apache.org/r/16269/#comment58258

Can we remove this statement about infinities? The code does not check for 
infinities, just 0 denoms. The point here is that zero divide is supposed to 
yield NULL.

It is worth thinking about whether infinity, NaN, etc. can be found in the 
stored data and if that causes any correctness issues elsewhere. 



ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NullUtil.java
https://reviews.apache.org/r/16269/#comment58257

Again, NaN is only for double, not long


- Eric Hanson


On Dec. 14, 2013, 1:38 a.m., Sergey Shelukhin wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/16269/
 ---
 
 (Updated Dec. 14, 2013, 1:38 a.m.)
 
 
 Review request for hive and Jitendra Pandey.
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 see JIRA
 
 
 Diffs
 -
 
   ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java a286024 
   ql/src/gen/vectorization/ExpressionTemplates/ColumnDivideColumn.txt 218ab6f 
   ql/src/gen/vectorization/ExpressionTemplates/ScalarDivideColumn.txt c969b2f 
   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NullUtil.java 
 e4ee372 
   ql/src/test/queries/clientpositive/vectorization_12.q 78af086 
   ql/src/test/queries/clientpositive/vectorization_14.q bc7730d 
   ql/src/test/results/clientpositive/vectorization_12.q.out 01e2506 
   ql/src/test/results/clientpositive/vectorization_14.q.out 247197a 
 
 Diff: https://reviews.apache.org/r/16269/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Sergey Shelukhin

[jira] [Commented] (HIVE-6034) vectorized % doesn't handle zeroes the same way as non-vectorized


[ 
https://issues.apache.org/jira/browse/HIVE-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849411#comment-13849411
 ] 

Eric Hanson commented on HIVE-6034:
---

Please see my comments on ReviewBoard

 vectorized % doesn't handle zeroes the same way as non-vectorized
 -

 Key: HIVE-6034
 URL: https://issues.apache.org/jira/browse/HIVE-6034
 Project: Hive
  Issue Type: Bug
  Components: Vectorization
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HIVE-6034.patch


 % 0 is NULL, but if vectorized it's NaN



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-5975) [WebHCat] templeton mapreduce job failed if provide define parameters

2013-12-16 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-5975:


   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Patch committed to trunk.
Thanks for the patch Shanyu, and thanks for the review Eugene!


 [WebHCat] templeton mapreduce job failed if provide define parameters
 ---

 Key: HIVE-5975
 URL: https://issues.apache.org/jira/browse/HIVE-5975
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 0.12.0, 0.13.0
Reporter: shanyu zhao
Assignee: shanyu zhao
 Fix For: 0.13.0

 Attachments: hive-5975.2.patch, hive-5975.patch


 Trying to submit a mapreduce job through templeton failed:
 curl -k -u user:pass -d user.name=user -d define=JobName=MRPiJob -d class=pi 
 -d arg=16 -d arg=100 -d jar=hadoop-mapreduce-examples.jar 
 https://xxx/templeton/v1/mapreduce/jar
 The error message is:
 Usage: org.apache.hadoop.examples.QuasiMonteCarlo nMaps nSamples
  Generic options supported are
  -conf configuration file specify an application configuration file
  -D property=value use value for given property
  -fs local|namenode:port specify a namenode
  -jt local|jobtracker:port specify a job tracker
  -files comma separated list of files specify comma separated files to be 
 copied to the map reduce cluster
  -libjars comma separated list of jars specify comma separated jar files to 
 include in the classpath.
  -archives comma separated list of archives specify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
  bin/hadoop command [genericOptions] [commandOptions]
 templeton: job failed with exit code 2
 Note that if we remove the define parameter it works fine.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Review Request 16299: HIVE-6013: Supporting Quoted Identifiers in Column Names

2013-12-16 Thread Harish Butani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16299/
---

Review request for hive, Ashutosh Chauhan and Alan Gates.


Bugs: HIVE-6013
https://issues.apache.org/jira/browse/HIVE-6013


Repository: hive-git


Description
---

Hive's current behavior on Quoted Identifiers is different from the normal 
interpretation. Quoted Identifier (using backticks) has a special 
interpretation for Select expressions(as Regular Expressions). Have documented 
current behavior and proposed a solution in attached doc.
Summary of solution is:
Introduce 'standard' quoted identifiers for columns only.
At the langauage level this is turned on by a flag.
At the metadata level we relax the constraint on column names.


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048 
  itests/qtest/pom.xml 8c249a0 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
3deed45 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java eb26e7f 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 321759b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 17e6aad 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 366b714 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1e6826f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ace1df9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/UnparseTranslator.java 8fe2262 
  ql/src/test/queries/clientnegative/invalid_columns.q f8be8c8 
  ql/src/test/queries/clientpositive/quotedid_alter.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_basic.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_partition.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_skew.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_smb.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_tblproperty.q PRE-CREATION 
  ql/src/test/results/clientnegative/invalid_columns.q.out 3311b0a 
  ql/src/test/results/clientpositive/quotedid_alter.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_basic.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_partition.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_skew.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_smb.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_tblproperty.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/16299/diff/


Testing
---

added new tests for create, alter, delete, query with columns containing 
special characters.
Tests start with quotedid


Thanks,

Harish Butani

[jira] [Updated] (HIVE-6013) Supporting Quoted Identifiers in Column Names


 [ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-6013:


Status: Patch Available  (was: Open)

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6013) Supporting Quoted Identifiers in Column Names


 [ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-6013:


Status: Open  (was: Patch Available)

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6013) Supporting Quoted Identifiers in Column Names


 [ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-6013:


Attachment: HIVE-6013.2.patch

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6021) Problem in GroupByOperator for handling distinct aggrgations


[ 
https://issues.apache.org/jira/browse/HIVE-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849593#comment-13849593
 ] 

Xuefu Zhang commented on HIVE-6021:
---

[~sunrui] Thanks for the update. Could you please attach your updated patch 
here? The review board has the latest thought.

 Problem in GroupByOperator for handling distinct aggrgations
 

 Key: HIVE-6021
 URL: https://issues.apache.org/jira/browse/HIVE-6021
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.12.0
Reporter: Sun Rui
Assignee: Sun Rui
 Attachments: HIVE-6021.1.patch


 Use the following test case with HIVE 0.12:
 {code:sql}
 create table src(key int, value string);
 load data local inpath 'src/data/files/kv1.txt' overwrite into table src;
 set hive.map.aggr=false; 
 select count(key),count(distinct value) from src group by key;
 {code}
 We will get an ArrayIndexOutOfBoundsException from GroupByOperator:
 {code}
 java.lang.RuntimeException: Error in configuring object
   at 
 org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
   at 
 org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
   at 
 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
   at 
 org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:485)
   at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
   at 
 org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
 Caused by: java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
   ... 5 more
 Caused by: java.lang.RuntimeException: Reduce operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:159)
   ... 10 more
 Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
   at 
 org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:281)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377)
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:152)
   ... 10 more
 {code}
 explain select count(key),count(distinct value) from src group by key;
 {code}
 STAGE PLANS:
   Stage: Stage-1
 Map Reduce
   Alias - Map Operator Tree:
 src 
   TableScan
 alias: src
 Select Operator
   expressions:
 expr: key
 type: int
 expr: value
 type: string
   outputColumnNames: key, value
   Reduce Output Operator
 key expressions:
   expr: key
   type: int
   expr: value
   type: string
 sort order: ++
 Map-reduce partition columns:
   expr: key
   type: int
 tag: -1
   Reduce Operator Tree:
 Group By Operator
   aggregations:
 expr: count(KEY._col0)   // The parameter causes this problem
^^^
 expr: count(DISTINCT KEY._col1:0._col0)
   bucketGroup: false
   keys:
 expr: KEY._col0
 type: int
   mode: complete
   outputColumnNames: _col0, _col1, _col2
   Select Operator
 expressions:
   expr: _col1
   type: bigint
   expr: _col2
   type: bigint
 outputColumnNames: _col0, _col1
 File Output Operator
   compressed: false
   GlobalTableId: 0
   table:
   input format: org.apache.hadoop.mapred.TextInputFormat
   output format: 
 org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
   Stage: Stage-0
 Fetch Operator
   limit: -1
 {code}
 The root cause is within GroupByOperator.initializeOp(). The method forgets 
 to handle the case:
 For a query has distinct aggregations, there is an aggregation function has a 
 parameter which is a groupby key column but not distinct key column.
 {code}
 if (unionExprEval != null) {
   String[] names = parameters.get(j).getExprString().split(\\.);
   //

Hive-trunk-hadoop2 - Build # 604 - Still Failing

Changes for Build #570
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #571
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #572
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #573
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #574
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #575
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #576

Changes for Build #577

Changes for Build #578

Changes for Build #579
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #580
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #581
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #582
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #583
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #584
[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)

[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #585
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)


Changes for Build #586
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #587
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore direct sql doesn't work with oracle (Sergey

Re: Review Request 16269: HIVE-6034 vectorized % doesn't handle zeroes the same way as non-vectorized

2013-12-16 Thread Sergey Shelukhin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16269/
---

(Updated Dec. 16, 2013, 7:18 p.m.)


Review request for hive and Jitendra Pandey.


Repository: hive-git


Description
---

see JIRA


Diffs (updated)
-

  ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java a286024 
  ql/src/gen/vectorization/ExpressionTemplates/ColumnDivideColumn.txt 218ab6f 
  ql/src/gen/vectorization/ExpressionTemplates/ScalarDivideColumn.txt c969b2f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NullUtil.java 
e4ee372 
  ql/src/test/queries/clientpositive/vectorization_12.q 78af086 
  ql/src/test/queries/clientpositive/vectorization_14.q bc7730d 
  ql/src/test/results/clientpositive/vectorization_12.q.out 01e2506 
  ql/src/test/results/clientpositive/vectorization_14.q.out 247197a 

Diff: https://reviews.apache.org/r/16269/diff/


Testing
---


Thanks,

Sergey Shelukhin

[jira] [Updated] (HIVE-6034) vectorized % doesn't handle zeroes the same way as non-vectorized

2013-12-16 Thread Sergey Shelukhin (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HIVE-6034:
---

Attachment: HIVE-6034.01.patch

Address RB feedback about javadocs

 vectorized % doesn't handle zeroes the same way as non-vectorized
 -

 Key: HIVE-6034
 URL: https://issues.apache.org/jira/browse/HIVE-6034
 Project: Hive
  Issue Type: Bug
  Components: Vectorization
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HIVE-6034.01.patch, HIVE-6034.patch


 % 0 is NULL, but if vectorized it's NaN



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-4256) JDBC2 HiveConnection does not use the specified database


[ 
https://issues.apache.org/jira/browse/HIVE-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849613#comment-13849613
 ] 

Prasad Mujumdar commented on HIVE-4256:
---

+1


 JDBC2 HiveConnection does not use the specified database
 

 Key: HIVE-4256
 URL: https://issues.apache.org/jira/browse/HIVE-4256
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, JDBC
Affects Versions: 0.11.0
Reporter: Chris Drome
Assignee: Anandha L Ranganathan
 Attachments: HIVE-4256.1.patch, HIVE-4256.2.patch, HIVE-4256.3.patch, 
 HIVE-4256.4.patch, HIVE-4256.5.patch, HIVE-4256.6.patch, HIVE-4256.patch


 HiveConnection ignores the database specified in the connection string when 
 configuring the connection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names


[ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849616#comment-13849616
 ] 

Eric Hanson commented on HIVE-6013:
---

The spec sounds reasonable. I'm fine with it. I'm curious though, did you 
consider allowing square brackets to delimit an identifier, like SQL Server 
allows? E.g. create table sale([Shipment Date] date, ); That seems a tad 
more readable than the back-quotes.

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6034) vectorized % doesn't handle zeroes the same way as non-vectorized


[ 
https://issues.apache.org/jira/browse/HIVE-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849619#comment-13849619
 ] 

Eric Hanson commented on HIVE-6034:
---

+1


 vectorized % doesn't handle zeroes the same way as non-vectorized
 -

 Key: HIVE-6034
 URL: https://issues.apache.org/jira/browse/HIVE-6034
 Project: Hive
  Issue Type: Bug
  Components: Vectorization
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HIVE-6034.01.patch, HIVE-6034.patch


 % 0 is NULL, but if vectorized it's NaN



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6036) A test case for embedded beeline - with URL jdbc:hive2:///default


[ 
https://issues.apache.org/jira/browse/HIVE-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849632#comment-13849632
 ] 

Prasad Mujumdar commented on HIVE-6036:
---

+1
Thanks for the addressing the issue!

 A test case for embedded beeline  -  with URL jdbc:hive2:///default
 ---

 Key: HIVE-6036
 URL: https://issues.apache.org/jira/browse/HIVE-6036
 Project: Hive
  Issue Type: Bug
Reporter: Anandha L Ranganathan
Assignee: Anandha L Ranganathan
 Attachments: HIVE-6036.patch


 A test case for embedded beeline would have been helpful. ie, with URL 
 jdbc:hive2:///default
 This causes beeline (JDBC driver) to invoken embedded hive.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Hive-trunk-h0.21 - Build # 2505 - Still Failing

Changes for Build #2471
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #2472
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #2473
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #2474
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #2475
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #2476
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #2477

Changes for Build #2478

Changes for Build #2479

Changes for Build #2480
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #2481

Changes for Build #2482
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #2483
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #2484
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #2485
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #2486
[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #2487
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)

[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)


Changes for Build #2488
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #2489
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore

[jira] [Updated] (HIVE-1466) Add NULL DEFINED AS to ROW FORMAT specification


 [ 
https://issues.apache.org/jira/browse/HIVE-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar updated HIVE-1466:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Patch committed to trunk.

 Add NULL DEFINED AS to ROW FORMAT specification
 ---

 Key: HIVE-1466
 URL: https://issues.apache.org/jira/browse/HIVE-1466
 Project: Hive
  Issue Type: Improvement
Reporter: Adam Kramer
Assignee: Prasad Mujumdar
 Attachments: HIVE-1466.1.patch, HIVE-1466.2.patch


 NULL values are passed to transformers as a literal backslash and a literal 
 N. NULL values are saved when INSERT OVERWRITing LOCAL DIRECTORies as NULL. 
 This is inconsistent.
 The ROW FORMAT specification of tables should be able to specify the manner 
 in which a null character is represented. ROW FORMAT NULL DEFINED AS '\N' or 
 '\003' or whatever should apply to all instances of table export and saving.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names


[ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849659#comment-13849659
 ] 

Hive QA commented on HIVE-6013:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618945/HIVE-6013.2.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4790 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_quotedid_skew
{noformat}

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/653/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/653/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618945

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5414) The result of show grant is not visible via JDBC

2013-12-16 Thread Phabricator (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849706#comment-13849706
 ] 

Phabricator commented on HIVE-5414:
---

thejas has commented on the revision HIVE-5414 [jira] The result of show grant 
is not visible via JDBC.

  The patch does not apply on trunk anymore. Can you also please rebase ?
  Sorry about the delay in reviewing this patch.

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/plan/ShowGrantDesc.java:36 I think 
property would be a better name for column, than key.
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java:1816 
can you also add test for column name and column type of result ?
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java:1814 
can you also add a test for 'show roles' ?

REVISION DETAIL
  https://reviews.facebook.net/D13209

To: JIRA, navis
Cc: Unknown Object (Phabricator User), thejas


 The result of show grant is not visible via JDBC
 

 Key: HIVE-5414
 URL: https://issues.apache.org/jira/browse/HIVE-5414
 Project: Hive
  Issue Type: Bug
  Components: Authorization, JDBC
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: D13209.1.patch, D13209.2.patch, D13209.3.patch, 
 HIVE-5414.4.patch.txt


 Currently, show grant / show role grant does not make fetch task, which 
 provides the result schema for jdbc clients.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Hive-trunk-h0.21 - Build # 2506 - Still Failing

Changes for Build #2471
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #2472
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #2473
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #2474
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #2475
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #2476
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #2477

Changes for Build #2478

Changes for Build #2479

Changes for Build #2480
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #2481

Changes for Build #2482
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #2483
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #2484
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #2485
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #2486
[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #2487
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)

[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)


Changes for Build #2488
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #2489
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore

[jira] [Commented] (HIVE-6034) vectorized % doesn't handle zeroes the same way as non-vectorized


[ 
https://issues.apache.org/jira/browse/HIVE-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849714#comment-13849714
 ] 

Hive QA commented on HIVE-6034:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618952/HIVE-6034.01.patch

{color:green}SUCCESS:{color} +1 4789 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/654/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/654/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618952

 vectorized % doesn't handle zeroes the same way as non-vectorized
 -

 Key: HIVE-6034
 URL: https://issues.apache.org/jira/browse/HIVE-6034
 Project: Hive
  Issue Type: Bug
  Components: Vectorization
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HIVE-6034.01.patch, HIVE-6034.patch


 % 0 is NULL, but if vectorized it's NaN



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-5812) HiveServer2 SSL connection transport binds to loopback address by default


 [ 
https://issues.apache.org/jira/browse/HIVE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar updated HIVE-5812:
--

Attachment: HIVE-5812.2.patch

Rebased patch

[~brocknoland] would you like to take another look ?

 HiveServer2 SSL connection transport binds to loopback address by default
 -

 Key: HIVE-5812
 URL: https://issues.apache.org/jira/browse/HIVE-5812
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
 Attachments: HIVE-5812.1.patch, HIVE-5812.2.patch


 The secure socket transport implemented as part of HIVE-5351, binds to 
 loopback address by default. If the bind interface gets used only if its 
 explicitly defined in the hive-site or via environment.
 This behavior should be same as non-SSL transport.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 16229: HIVE-6010 create a test that would ensure vectorization produces same results as non-vectorized execution

2013-12-16 Thread Jitendra Pandey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16229/#review30464
---



ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java
https://reviews.apache.org/r/16229/#comment58282

We should throw an exception if it is null. What is the reason for the 
change in behavior?



ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java
https://reviews.apache.org/r/16229/#comment58283

Is it ok to write to stdout?



ql/src/test/queries/clientcompare/vectorized_math_funcs_00.qv
https://reviews.apache.org/r/16229/#comment58288

The name of the file could be suffixed to indicate vectorization disabled 
or enabled.


- Jitendra Pandey


On Dec. 13, 2013, 7:15 p.m., Sergey Shelukhin wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/16229/
 ---
 
 (Updated Dec. 13, 2013, 7:15 p.m.)
 
 
 Review request for hive and Jitendra Pandey.
 
 
 Bugs: HIVE-6010
 https://issues.apache.org/jira/browse/HIVE-6010
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 See jira.
 
 
 Diffs
 -
 
   ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java 85351aa 
   itests/qtest/pom.xml 8c249a0 
   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java c16e82d 
   ql/src/test/queries/clientcompare/vectorized_math_funcs.q PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_00.qv PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_01.qv PRE-CREATION 
   ql/src/test/templates/TestCompareCliDriver.vm PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/16229/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Sergey Shelukhin

[jira] [Updated] (HIVE-5992) Hive inconsistently converts timestamp in AVG and SUM UDAF's


 [ 
https://issues.apache.org/jira/browse/HIVE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-5992:
--

Attachment: HIVE-5992.patch

 Hive inconsistently converts timestamp in AVG and SUM UDAF's
 

 Key: HIVE-5992
 URL: https://issues.apache.org/jira/browse/HIVE-5992
 Project: Hive
  Issue Type: Bug
  Components: UDF
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-5992.patch


 {code}
 hive select t, sum(t), count(*), sum(t)/count(*), avg(t) from ts group by t;
 ...
 OK
 1977-03-15 12:34:22.345678 227306062  1  227306062
 2.27306062345678E8
 {code}
 As it can be seen, timestamp value (1977-03-15 12:34:22.345678) is converted 
 with fractional part ignored in sum, while preserved in avg. As a further 
 result, sum()/count() is not equivalent to avg.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-5992) Hive inconsistently converts timestamp in AVG and SUM UDAF's


 [ 
https://issues.apache.org/jira/browse/HIVE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-5992:
--

Status: Patch Available  (was: Open)

 Hive inconsistently converts timestamp in AVG and SUM UDAF's
 

 Key: HIVE-5992
 URL: https://issues.apache.org/jira/browse/HIVE-5992
 Project: Hive
  Issue Type: Bug
  Components: UDF
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-5992.patch


 {code}
 hive select t, sum(t), count(*), sum(t)/count(*), avg(t) from ts group by t;
 ...
 OK
 1977-03-15 12:34:22.345678 227306062  1  227306062
 2.27306062345678E8
 {code}
 As it can be seen, timestamp value (1977-03-15 12:34:22.345678) is converted 
 with fractional part ignored in sum, while preserved in avg. As a further 
 result, sum()/count() is not equivalent to avg.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names

[
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849765#comment-13849765
]

Harish Butani commented on HIVE-6013:
-

Back-ticks are already there (sort of). For e.g: today view definitions are
unparsed(and persisted) to wrap all identifiers with back-ticks.
Moving to square brackets would either require a migration step, or supporting
both back-ticks and square brackets. Don't want to take on either option in
this first step.
Besides not clear which is the preferred style for the hive community:
back-ticks seems to have come from the mysql way of specifying quoted
identifiers.

Supporting Quoted Identifiers in Column Names
-

Key: HIVE-6013
URL: https://issues.apache.org/jira/browse/HIVE-6013
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
Fix For: 0.13.0

Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch,
QuotedIdentifier.html

Hive's current behavior on Quoted Identifiers is different from the normal
interpretation. Quoted Identifier (using backticks) has a special
interpretation for Select expressions(as Regular Expressions). Have
documented current behavior and proposed a solution in attached doc.
Summary of solution is:
- Introduce 'standard' quoted identifiers for columns only.
- At the langauage level this is turned on by a flag.
- At the metadata level we relax the constraint on column names.

--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5812) HiveServer2 SSL connection transport binds to loopback address by default


[ 
https://issues.apache.org/jira/browse/HIVE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849770#comment-13849770
 ] 

Hive QA commented on HIVE-5812:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618972/HIVE-5812.2.patch

{color:green}SUCCESS:{color} +1 4789 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/655/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/655/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618972

 HiveServer2 SSL connection transport binds to loopback address by default
 -

 Key: HIVE-5812
 URL: https://issues.apache.org/jira/browse/HIVE-5812
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
 Attachments: HIVE-5812.1.patch, HIVE-5812.2.patch


 The secure socket transport implemented as part of HIVE-5351, binds to 
 loopback address by default. If the bind interface gets used only if its 
 explicitly defined in the hive-site or via environment.
 This behavior should be same as non-SSL transport.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5795) Hive should be able to skip header and footer rows when reading data file for a table

2013-12-16 Thread Shuaishuai Nie (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849774#comment-13849774
 ] 

Shuaishuai Nie commented on HIVE-5795:
--

Sorry, wrong link to the new review board. Update the link here: 
https://reviews.apache.org/r/16184/

 Hive should be able to skip header and footer rows when reading data file for 
 a table
 -

 Key: HIVE-5795
 URL: https://issues.apache.org/jira/browse/HIVE-5795
 Project: Hive
  Issue Type: Bug
Reporter: Shuaishuai Nie
Assignee: Shuaishuai Nie
 Attachments: HIVE-5795.1.patch, HIVE-5795.2.patch


 Hive should be able to skip header and footer lines when reading data file 
 from table. In this way, user don't need to processing data which generated 
 by other application with a header or footer and directly use the file for 
 table operations.
 To implement this, the idea is adding new properties in table descriptions to 
 define the number of lines in header and footer and skip them when reading 
 the record from record reader. An DDL example for creating a table with 
 header and footer should be like this:
 {code}
 Create external table testtable (name string, message string) row format 
 delimited fields terminated by '\t' lines terminated by '\n' location 
 '/testtable' tblproperties (skip.header.number=1, 
 skip.footer.number=2);
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Created] (HIVE-6039) Round, AVG and SUM functions reject char/varch input while accepting string input

Xuefu Zhang created HIVE-6039:
-

 Summary: Round, AVG and SUM functions reject char/varch input 
while accepting string input
 Key: HIVE-6039
 URL: https://issues.apache.org/jira/browse/HIVE-6039
 Project: Hive
  Issue Type: Bug
  Components: UDF
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: 0.13.0


An error similar to the following will occur:
{code}
hive create table tabs (c char(8), vc varchar(10)) ROW FORMAT DELIMITED FIELDS 
TERMINATED BY '\t';
hive select sum(c), avg(c), sum(vc), avg(vc) from tabs;
FAILED: UDFArgumentTypeException Only numeric or string type arguments are 
accepted but char(8) is passed.
{code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Hive-trunk-hadoop2 - Build # 605 - Still Failing

Changes for Build #570
[rhbutani] HIVE-5849 Improve the stats of operators based on heuristics in the 
absence of any column statistics (Prasanth Jayachandran via Harish Butani)

[hashutosh] HIVE-5793 : Update hive-default.xml.template for HIVE4002 (Navis 
via Ashutosh Chauhan)


Changes for Build #571
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #572
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #573
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #574
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #575
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #576

Changes for Build #577

Changes for Build #578

Changes for Build #579
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #580
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #581
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #582
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #583
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #584
[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)

[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #585
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)


Changes for Build #586
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #587
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore direct sql doesn't work with oracle (Sergey

[jira] [Updated] (HIVE-6013) Supporting Quoted Identifiers in Column Names


 [ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-6013:


Status: Open  (was: Patch Available)

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6013) Supporting Quoted Identifiers in Column Names


 [ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-6013:


Status: Patch Available  (was: Open)

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6013) Supporting Quoted Identifiers in Column Names


 [ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-6013:


Attachment: HIVE-6013.3.patch

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 16299: HIVE-6013: Supporting Quoted Identifiers in Column Names

2013-12-16 Thread Harish Butani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16299/
---

(Updated Dec. 16, 2013, 10:22 p.m.)


Review request for hive, Ashutosh Chauhan and Alan Gates.


Changes
---

fix paths in quotedid_skew.q


Bugs: HIVE-6013
https://issues.apache.org/jira/browse/HIVE-6013


Repository: hive-git


Description
---

Hive's current behavior on Quoted Identifiers is different from the normal 
interpretation. Quoted Identifier (using backticks) has a special 
interpretation for Select expressions(as Regular Expressions). Have documented 
current behavior and proposed a solution in attached doc.
Summary of solution is:
Introduce 'standard' quoted identifiers for columns only.
At the langauage level this is turned on by a flag.
At the metadata level we relax the constraint on column names.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048 
  itests/qtest/pom.xml 8c249a0 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
3deed45 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java eb26e7f 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 321759b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 17e6aad 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g ed9917d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1e6826f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d18ea03 
  ql/src/java/org/apache/hadoop/hive/ql/parse/UnparseTranslator.java 8fe2262 
  ql/src/test/queries/clientnegative/invalid_columns.q f8be8c8 
  ql/src/test/queries/clientpositive/quotedid_alter.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_basic.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_partition.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_skew.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_smb.q PRE-CREATION 
  ql/src/test/queries/clientpositive/quotedid_tblproperty.q PRE-CREATION 
  ql/src/test/results/clientnegative/invalid_columns.q.out 3311b0a 
  ql/src/test/results/clientpositive/quotedid_alter.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_basic.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_partition.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_skew.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_smb.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/quotedid_tblproperty.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/16299/diff/


Testing
---

added new tests for create, alter, delete, query with columns containing 
special characters.
Tests start with quotedid


Thanks,

Harish Butani

Re: Review Request 16184: Hive should be able to skip header and footer rows when reading data file for a table (HIVE-5795)


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16184/#review30489
---



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/16184/#comment58323

fix comment to say max number of lines of footer user can set for a table 
file



ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
https://reviews.apache.org/r/16184/#comment58324

Please use block comment style for multi-line comments. Please use full 
sentences and proper punctuation (start with capital letter, end with period).


- Eric Hanson


On Dec. 11, 2013, 9:19 p.m., Shuaishuai Nie wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/16184/
 ---
 
 (Updated Dec. 11, 2013, 9:19 p.m.)
 
 
 Review request for hive, Eric Hanson and Thejas Nair.
 
 
 Bugs: hive-5795
 https://issues.apache.org/jira/browse/hive-5795
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 Hive should be able to skip header and footer rows when reading data file for 
 a table
 (follow up with review https://reviews.apache.org/r/15663/diff/#index_header)
 
 
 Diffs
 -
 
   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048 
   conf/hive-default.xml.template c61a0bb 
   data/files/header_footer_table_1/0001.txt PRE-CREATION 
   data/files/header_footer_table_1/0002.txt PRE-CREATION 
   data/files/header_footer_table_1/0003.txt PRE-CREATION 
   data/files/header_footer_table_2/2012/01/01/0001.txt PRE-CREATION 
   data/files/header_footer_table_2/2012/01/02/0002.txt PRE-CREATION 
   data/files/header_footer_table_2/2012/01/03/0003.txt PRE-CREATION 
   itests/qtest/pom.xml c3cbb89 
   ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java d2b2526 
   ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 
 dd5cb6b 
   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 974a5d6 
   
 ql/src/test/org/apache/hadoop/hive/ql/io/TestHiveBinarySearchRecordReader.java
  85dd975 
   ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java 
 0686d9b 
   ql/src/test/queries/clientnegative/file_with_header_footer_negative.q 
 PRE-CREATION 
   ql/src/test/queries/clientpositive/file_with_header_footer.q PRE-CREATION 
   ql/src/test/results/clientnegative/file_with_header_footer_negative.q.out 
 PRE-CREATION 
   ql/src/test/results/clientpositive/file_with_header_footer.q.out 
 PRE-CREATION 
   serde/if/serde.thrift 2ceb572 
   
 serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
  22a6168 
 
 Diff: https://reviews.apache.org/r/16184/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Shuaishuai Nie

[jira] [Commented] (HIVE-5795) Hive should be able to skip header and footer rows when reading data file for a table


[ 
https://issues.apache.org/jira/browse/HIVE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849801#comment-13849801
 ] 

Eric Hanson commented on HIVE-5795:
---

Looks good, but there are a bunch of places where you need to modify your 
comments to obey Hive coding conventions -- 
https://cwiki.apache.org/confluence/display/Hive/HowToContribute --, especially 
for putting blank lines before comments, and also in a small number of places 
putting blanks between tokens.

 Hive should be able to skip header and footer rows when reading data file for 
 a table
 -

 Key: HIVE-5795
 URL: https://issues.apache.org/jira/browse/HIVE-5795
 Project: Hive
  Issue Type: Bug
Reporter: Shuaishuai Nie
Assignee: Shuaishuai Nie
 Attachments: HIVE-5795.1.patch, HIVE-5795.2.patch


 Hive should be able to skip header and footer lines when reading data file 
 from table. In this way, user don't need to processing data which generated 
 by other application with a header or footer and directly use the file for 
 table operations.
 To implement this, the idea is adding new properties in table descriptions to 
 define the number of lines in header and footer and skip them when reading 
 the record from record reader. An DDL example for creating a table with 
 header and footer should be like this:
 {code}
 Create external table testtable (name string, message string) row format 
 delimited fields terminated by '\t' lines terminated by '\n' location 
 '/testtable' tblproperties (skip.header.number=1, 
 skip.footer.number=2);
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names


[ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849802#comment-13849802
 ] 

Eric Hanson commented on HIVE-6013:
---

Okay, sounds fine then.

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names

2013-12-16 Thread Sergey Shelukhin (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849809#comment-13849809
 ] 

Sergey Shelukhin commented on HIVE-6013:


The fun fact is that double quotes are ANSI way of specifying quoted 
identifiers :)

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6039) Round, AVG and SUM functions reject char/varch input while accepting string input


 [ 
https://issues.apache.org/jira/browse/HIVE-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6039:
--

Attachment: HIVE-6039.patch

 Round, AVG and SUM functions reject char/varch input while accepting string 
 input
 -

 Key: HIVE-6039
 URL: https://issues.apache.org/jira/browse/HIVE-6039
 Project: Hive
  Issue Type: Bug
  Components: UDF
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: 0.13.0

 Attachments: HIVE-6039.patch


 An error similar to the following will occur:
 {code}
 hive create table tabs (c char(8), vc varchar(10)) ROW FORMAT DELIMITED 
 FIELDS TERMINATED BY '\t';
 hive select sum(c), avg(c), sum(vc), avg(vc) from tabs;
 FAILED: UDFArgumentTypeException Only numeric or string type arguments are 
 accepted but char(8) is passed.
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6039) Round, AVG and SUM functions reject char/varch input while accepting string input


 [ 
https://issues.apache.org/jira/browse/HIVE-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6039:
--

Status: Patch Available  (was: Open)

 Round, AVG and SUM functions reject char/varch input while accepting string 
 input
 -

 Key: HIVE-6039
 URL: https://issues.apache.org/jira/browse/HIVE-6039
 Project: Hive
  Issue Type: Bug
  Components: UDF
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: 0.13.0

 Attachments: HIVE-6039.patch


 An error similar to the following will occur:
 {code}
 hive create table tabs (c char(8), vc varchar(10)) ROW FORMAT DELIMITED 
 FIELDS TERMINATED BY '\t';
 hive select sum(c), avg(c), sum(vc), avg(vc) from tabs;
 FAILED: UDFArgumentTypeException Only numeric or string type arguments are 
 accepted but char(8) is passed.
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5992) Hive inconsistently converts timestamp in AVG and SUM UDAF's


[ 
https://issues.apache.org/jira/browse/HIVE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849819#comment-13849819
 ] 

Hive QA commented on HIVE-5992:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618977/HIVE-5992.patch

{color:green}SUCCESS:{color} +1 4789 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/656/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/656/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618977

 Hive inconsistently converts timestamp in AVG and SUM UDAF's
 

 Key: HIVE-5992
 URL: https://issues.apache.org/jira/browse/HIVE-5992
 Project: Hive
  Issue Type: Bug
  Components: UDF
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-5992.patch


 {code}
 hive select t, sum(t), count(*), sum(t)/count(*), avg(t) from ts group by t;
 ...
 OK
 1977-03-15 12:34:22.345678 227306062  1  227306062
 2.27306062345678E8
 {code}
 As it can be seen, timestamp value (1977-03-15 12:34:22.345678) is converted 
 with fractional part ignored in sum, while preserved in avg. As a further 
 result, sum()/count() is not equivalent to avg.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-5230) Better error reporting by async threads in HiveServer2

2013-12-16 Thread Vaibhav Gumashta (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-5230:
---

Attachment: HIVE-5230.10.patch

Patch rebased on latest trunk.

 Better error reporting by async threads in HiveServer2
 --

 Key: HIVE-5230
 URL: https://issues.apache.org/jira/browse/HIVE-5230
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0, 0.13.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
 Fix For: 0.13.0

 Attachments: HIVE-5230.1.patch, HIVE-5230.1.patch, 
 HIVE-5230.10.patch, HIVE-5230.2.patch, HIVE-5230.3.patch, HIVE-5230.4.patch, 
 HIVE-5230.6.patch, HIVE-5230.7.patch, HIVE-5230.8.patch, HIVE-5230.9.patch


 [HIVE-4617|https://issues.apache.org/jira/browse/HIVE-4617] provides support 
 for async execution in HS2. When a background thread gets an error, currently 
 the client can only poll for the operation state and also the error with its 
 stacktrace is logged. However, it will be useful to provide a richer error 
 response like thrift API does with TStatus (which is constructed while 
 building a Thrift response object). 



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5230) Better error reporting by async threads in HiveServer2

2013-12-16 Thread Vaibhav Gumashta (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849827#comment-13849827
 ] 

Vaibhav Gumashta commented on HIVE-5230:


[~thejas] Uploaded a rebased patch which contains the thrift generated code as 
well. Thanks!

 Better error reporting by async threads in HiveServer2
 --

 Key: HIVE-5230
 URL: https://issues.apache.org/jira/browse/HIVE-5230
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0, 0.13.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
 Fix For: 0.13.0

 Attachments: HIVE-5230.1.patch, HIVE-5230.1.patch, 
 HIVE-5230.10.patch, HIVE-5230.2.patch, HIVE-5230.3.patch, HIVE-5230.4.patch, 
 HIVE-5230.6.patch, HIVE-5230.7.patch, HIVE-5230.8.patch, HIVE-5230.9.patch


 [HIVE-4617|https://issues.apache.org/jira/browse/HIVE-4617] provides support 
 for async execution in HS2. When a background thread gets an error, currently 
 the client can only poll for the operation state and also the error with its 
 stacktrace is logged. However, it will be useful to provide a richer error 
 response like thrift API does with TStatus (which is constructed while 
 building a Thrift response object). 



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-5230) Better error reporting by async threads in HiveServer2

2013-12-16 Thread Vaibhav Gumashta (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-5230:
---

Status: Patch Available  (was: Open)

 Better error reporting by async threads in HiveServer2
 --

 Key: HIVE-5230
 URL: https://issues.apache.org/jira/browse/HIVE-5230
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0, 0.13.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
 Fix For: 0.13.0

 Attachments: HIVE-5230.1.patch, HIVE-5230.1.patch, 
 HIVE-5230.10.patch, HIVE-5230.2.patch, HIVE-5230.3.patch, HIVE-5230.4.patch, 
 HIVE-5230.6.patch, HIVE-5230.7.patch, HIVE-5230.8.patch, HIVE-5230.9.patch


 [HIVE-4617|https://issues.apache.org/jira/browse/HIVE-4617] provides support 
 for async execution in HS2. When a background thread gets an error, currently 
 the client can only poll for the operation state and also the error with its 
 stacktrace is logged. However, it will be useful to provide a richer error 
 response like thrift API does with TStatus (which is constructed while 
 building a Thrift response object). 



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5812) HiveServer2 SSL connection transport binds to loopback address by default

2013-12-16 Thread Brock Noland (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849843#comment-13849843
 ] 

Brock Noland commented on HIVE-5812:


+1

 HiveServer2 SSL connection transport binds to loopback address by default
 -

 Key: HIVE-5812
 URL: https://issues.apache.org/jira/browse/HIVE-5812
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
 Attachments: HIVE-5812.1.patch, HIVE-5812.2.patch


 The secure socket transport implemented as part of HIVE-5351, binds to 
 loopback address by default. If the bind interface gets used only if its 
 explicitly defined in the hive-site or via environment.
 This behavior should be same as non-SSL transport.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 16229: HIVE-6010 create a test that would ensure vectorization produces same results as non-vectorized execution

2013-12-16 Thread Sergey Shelukhin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16229/#review30486
---



ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java
https://reviews.apache.org/r/16229/#comment58320

that is because the new test has no result directory, it compares outputs 
of multiple queries



ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java
https://reviews.apache.org/r/16229/#comment58321

yeah, it ends up being written to build output. Do you think it should be 
written elsewhere?



ql/src/test/queries/clientcompare/vectorized_math_funcs_00.qv
https://reviews.apache.org/r/16229/#comment58355

this will muddy the naming convention for versions and potentially cause 
ambiguity; discussed the details in person


- Sergey Shelukhin


On Dec. 13, 2013, 7:15 p.m., Sergey Shelukhin wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/16229/
 ---
 
 (Updated Dec. 13, 2013, 7:15 p.m.)
 
 
 Review request for hive and Jitendra Pandey.
 
 
 Bugs: HIVE-6010
 https://issues.apache.org/jira/browse/HIVE-6010
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 See jira.
 
 
 Diffs
 -
 
   ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java 85351aa 
   itests/qtest/pom.xml 8c249a0 
   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java c16e82d 
   ql/src/test/queries/clientcompare/vectorized_math_funcs.q PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_00.qv PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_01.qv PRE-CREATION 
   ql/src/test/templates/TestCompareCliDriver.vm PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/16229/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Sergey Shelukhin

Re: Review Request 16229: HIVE-6010 create a test that would ensure vectorization produces same results as non-vectorized execution

2013-12-16 Thread Sergey Shelukhin



 On Dec. 16, 2013, 9:19 p.m., Jitendra Pandey wrote:
  ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java, line 453
  https://reviews.apache.org/r/16229/diff/2/?file=397661#file397661line453
 
  Is it ok to write to stdout?

yes, it ends up in build output


- Sergey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16229/#review30464
---


On Dec. 13, 2013, 7:15 p.m., Sergey Shelukhin wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/16229/
 ---
 
 (Updated Dec. 13, 2013, 7:15 p.m.)
 
 
 Review request for hive and Jitendra Pandey.
 
 
 Bugs: HIVE-6010
 https://issues.apache.org/jira/browse/HIVE-6010
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 See jira.
 
 
 Diffs
 -
 
   ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java 85351aa 
   itests/qtest/pom.xml 8c249a0 
   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java c16e82d 
   ql/src/test/queries/clientcompare/vectorized_math_funcs.q PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_00.qv PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_01.qv PRE-CREATION 
   ql/src/test/templates/TestCompareCliDriver.vm PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/16229/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Sergey Shelukhin

[jira] [Commented] (HIVE-6039) Round, AVG and SUM functions reject char/varch input while accepting string input


[ 
https://issues.apache.org/jira/browse/HIVE-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849869#comment-13849869
 ] 

Hive QA commented on HIVE-6039:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618986/HIVE-6039.patch

{color:green}SUCCESS:{color} +1 4790 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/658/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/658/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618986

 Round, AVG and SUM functions reject char/varch input while accepting string 
 input
 -

 Key: HIVE-6039
 URL: https://issues.apache.org/jira/browse/HIVE-6039
 Project: Hive
  Issue Type: Bug
  Components: UDF
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: 0.13.0

 Attachments: HIVE-6039.patch


 An error similar to the following will occur:
 {code}
 hive create table tabs (c char(8), vc varchar(10)) ROW FORMAT DELIMITED 
 FIELDS TERMINATED BY '\t';
 hive select sum(c), avg(c), sum(vc), avg(vc) from tabs;
 FAILED: UDFArgumentTypeException Only numeric or string type arguments are 
 accepted but char(8) is passed.
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6017) Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


 [ 
https://issues.apache.org/jira/browse/HIVE-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Hanson updated HIVE-6017:
--

Attachment: HIVE-6017.01.patch

 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 ---

 Key: HIVE-6017
 URL: https://issues.apache.org/jira/browse/HIVE-6017
 Project: Hive
  Issue Type: Sub-task
Reporter: Eric Hanson
Assignee: Eric Hanson
 Attachments: HIVE-6017.01.patch


 Contribute the Decimal128 high-performance decimal package developed by 
 Microsoft to Hive. This was originally written for Microsoft PolyBase by 
 Hideaki Kimura.
 This code is about 8X more efficient than Java BigDecimal for typical 
 operations. It uses a finite (128 bit) precision and can handle up to 
 decimal(38, X). It is also mutable so you can change the contents of an 
 existing object. This helps reduce the cost of new() and garbage collection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6017) Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


 [ 
https://issues.apache.org/jira/browse/HIVE-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Hanson updated HIVE-6017:
--

Affects Version/s: 0.13.0
   Status: Patch Available  (was: In Progress)

Added Decimal128 package into Hive package org.apache.hadoop.hive.common.type. 
Verified that it compiles and unit tests pass. 

 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 ---

 Key: HIVE-6017
 URL: https://issues.apache.org/jira/browse/HIVE-6017
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Eric Hanson
 Attachments: HIVE-6017.01.patch


 Contribute the Decimal128 high-performance decimal package developed by 
 Microsoft to Hive. This was originally written for Microsoft PolyBase by 
 Hideaki Kimura.
 This code is about 8X more efficient than Java BigDecimal for typical 
 operations. It uses a finite (128 bit) precision and can handle up to 
 decimal(38, X). It is also mutable so you can change the contents of an 
 existing object. This helps reduce the cost of new() and garbage collection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF

2013-12-16 Thread Mohammad Islam


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/
---

(Updated Dec. 17, 2013, midnight)


Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.


Changes
---

Fix the failed test case.


Bugs: HIVE-5829
https://issues.apache.org/jira/browse/HIVE-5829


Repository: hive-git


Description
---

Rewrite the UDFS *pads and *trim using GenericUDF.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
bca1f26 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java 
eff251f 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFLTrim.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFLpad.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFRTrim.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFRpad.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFTrim.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/15654/diff/


Testing
---


Thanks,

Mohammad Islam

[jira] [Updated] (HIVE-5829) Rewrite Trim and Pad UDFs based on GenericUDF

2013-12-16 Thread Mohammad Kamrul Islam (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mohammad Kamrul Islam updated HIVE-5829:


Attachment: HIVE-5829.2.patch
tmp.HIVE-5829.patch

Addressed the failed test case and rebased with latest code base.

 Rewrite Trim and Pad UDFs based on GenericUDF
 -

 Key: HIVE-5829
 URL: https://issues.apache.org/jira/browse/HIVE-5829
 Project: Hive
  Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: HIVE-5829.1.patch, HIVE-5829.2.patch, tmp.HIVE-5829.patch


 This JIRA includes following UDFs:
 1. trim()
 2. ltrim()
 3. rtrim()
 4. lpad()
 5. rpad()



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Review Request 16307: Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16307/
---

Review request for hive and Gunther Hagleitner.


Bugs: HIVE-6017
https://issues.apache.org/jira/browse/HIVE-6017


Repository: hive-git


Description
---

Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to 
Hive


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SignedInt128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SqlMathUtil.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/Decimal128Test.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/SignedInt128Test.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/SqlMathUtilTest.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/UnsignedInt128Test.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/16307/diff/


Testing
---


Thanks,

Eric Hanson

[jira] [Commented] (HIVE-6015) vectorized logarithm produces results for 0 that are different from a non-vectorized one

2013-12-16 Thread Sergey Shelukhin (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849896#comment-13849896
 ] 

Sergey Shelukhin commented on HIVE-6015:


[~ehans] can you commit? I don't have access

 vectorized logarithm produces results for 0 that are different from a 
 non-vectorized one
 

 Key: HIVE-6015
 URL: https://issues.apache.org/jira/browse/HIVE-6015
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
  Labels: vectorization
 Attachments: HIVE-6015.patch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6017) Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


[ 
https://issues.apache.org/jira/browse/HIVE-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849901#comment-13849901
 ] 

Eric Hanson commented on HIVE-6017:
---

Code review available at https://reviews.apache.org/r/16307/

 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 ---

 Key: HIVE-6017
 URL: https://issues.apache.org/jira/browse/HIVE-6017
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Eric Hanson
 Attachments: HIVE-6017.01.patch


 Contribute the Decimal128 high-performance decimal package developed by 
 Microsoft to Hive. This was originally written for Microsoft PolyBase by 
 Hideaki Kimura.
 This code is about 8X more efficient than Java BigDecimal for typical 
 operations. It uses a finite (128 bit) precision and can handle up to 
 decimal(38, X). It is also mutable so you can change the contents of an 
 existing object. This helps reduce the cost of new() and garbage collection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 16307: Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive

2013-12-16 Thread Brock Noland


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16307/#review30503
---


FYI PTest required tests start with Test not end with it.

- Brock Noland


On Dec. 17, 2013, 12:01 a.m., Eric Hanson wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/16307/
 ---
 
 (Updated Dec. 17, 2013, 12:01 a.m.)
 
 
 Review request for hive and Gunther Hagleitner.
 
 
 Bugs: HIVE-6017
 https://issues.apache.org/jira/browse/HIVE-6017
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 
 
 Diffs
 -
 
   common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 
 PRE-CREATION 
   common/src/java/org/apache/hadoop/hive/common/type/SignedInt128.java 
 PRE-CREATION 
   common/src/java/org/apache/hadoop/hive/common/type/SqlMathUtil.java 
 PRE-CREATION 
   common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java 
 PRE-CREATION 
   common/src/test/org/apache/hadoop/hive/common/type/Decimal128Test.java 
 PRE-CREATION 
   common/src/test/org/apache/hadoop/hive/common/type/SignedInt128Test.java 
 PRE-CREATION 
   common/src/test/org/apache/hadoop/hive/common/type/SqlMathUtilTest.java 
 PRE-CREATION 
   common/src/test/org/apache/hadoop/hive/common/type/UnsignedInt128Test.java 
 PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/16307/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Eric Hanson

[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names


[ 
https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849923#comment-13849923
 ] 

Hive QA commented on HIVE-6013:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618984/HIVE-6013.3.patch

{color:green}SUCCESS:{color} +1 4793 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/659/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/659/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618984

 Supporting Quoted Identifiers in Column Names
 -

 Key: HIVE-6013
 URL: https://issues.apache.org/jira/browse/HIVE-6013
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Harish Butani
Assignee: Harish Butani
 Fix For: 0.13.0

 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, 
 QuotedIdentifier.html


 Hive's current behavior on Quoted Identifiers is different from the normal 
 interpretation. Quoted Identifier (using backticks) has a special 
 interpretation for Select expressions(as Regular Expressions). Have 
 documented current behavior and proposed a solution in attached doc.
 Summary of solution is:
 - Introduce 'standard' quoted identifiers for columns only. 
 - At the langauage level this is turned on by a flag.
 - At the metadata level we relax the constraint on column names.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 16229: HIVE-6010 create a test that would ensure vectorization produces same results as non-vectorized execution

2013-12-16 Thread Jitendra Pandey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16229/#review30506
---

Ship it!


Ship It!

- Jitendra Pandey


On Dec. 13, 2013, 7:15 p.m., Sergey Shelukhin wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/16229/
 ---
 
 (Updated Dec. 13, 2013, 7:15 p.m.)
 
 
 Review request for hive and Jitendra Pandey.
 
 
 Bugs: HIVE-6010
 https://issues.apache.org/jira/browse/HIVE-6010
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 See jira.
 
 
 Diffs
 -
 
   ant/src/org/apache/hadoop/hive/ant/QTestGenTask.java 85351aa 
   itests/qtest/pom.xml 8c249a0 
   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java c16e82d 
   ql/src/test/queries/clientcompare/vectorized_math_funcs.q PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_00.qv PRE-CREATION 
   ql/src/test/queries/clientcompare/vectorized_math_funcs_01.qv PRE-CREATION 
   ql/src/test/templates/TestCompareCliDriver.vm PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/16229/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Sergey Shelukhin

[jira] [Updated] (HIVE-6015) vectorized logarithm produces results for 0 that are different from a non-vectorized one


 [ 
https://issues.apache.org/jira/browse/HIVE-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Hanson updated HIVE-6015:
--

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

 vectorized logarithm produces results for 0 that are different from a 
 non-vectorized one
 

 Key: HIVE-6015
 URL: https://issues.apache.org/jira/browse/HIVE-6015
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
  Labels: vectorization
 Fix For: 0.13.0

 Attachments: HIVE-6015.patch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6010) create a test that would ensure vectorization produces same results as non-vectorized execution

2013-12-16 Thread Jitendra Nath Pandey (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849926#comment-13849926
 ] 

Jitendra Nath Pandey commented on HIVE-6010:


+1

 create a test that would ensure vectorization produces same results as 
 non-vectorized execution
 ---

 Key: HIVE-6010
 URL: https://issues.apache.org/jira/browse/HIVE-6010
 Project: Hive
  Issue Type: Test
  Components: Tests, Vectorization
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: HIVE-6010.01.patch, HIVE-6010.02.patch, HIVE-6010.patch


 So as to ensure that vectorization is not forgotten when changes are made to 
 things. Obviously it would not be viable to have a bulletproof test, but at 
 least a subset of operations can be verified.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6015) vectorized logarithm produces results for 0 that are different from a non-vectorized one


[ 
https://issues.apache.org/jira/browse/HIVE-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849927#comment-13849927
 ] 

Eric Hanson commented on HIVE-6015:
---

Committed to trunk. Thanks Sergey!

 vectorized logarithm produces results for 0 that are different from a 
 non-vectorized one
 

 Key: HIVE-6015
 URL: https://issues.apache.org/jira/browse/HIVE-6015
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
  Labels: vectorization
 Fix For: 0.13.0

 Attachments: HIVE-6015.patch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-5973) SMB joins produce incorrect results with multiple partitions and buckets


 [ 
https://issues.apache.org/jira/browse/HIVE-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-5973:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

thanks Vikram

 SMB joins produce incorrect results with multiple partitions and buckets
 

 Key: HIVE-5973
 URL: https://issues.apache.org/jira/browse/HIVE-5973
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.13.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
 Fix For: 0.13.0

 Attachments: HIVE-5973.1.patch, HIVE-5973.2.patch


 It looks like there is an issue with re-using the output object array in the 
 select operator. When we read rows of the non-big tables, we hold on to the 
 output object in the priority queue. This causes hive to produce incorrect 
 results because all the elements in the priority queue refer to the same 
 object and the join happens on only one of the buckets.
 {noformat}
 output[i] = eval[i].evaluate(row);
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 16307: Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16307/
---

(Updated Dec. 17, 2013, 1:03 a.m.)


Review request for hive and Gunther Hagleitner.


Changes
---

Fixed test class names to start with Test.


Bugs: HIVE-6017
https://issues.apache.org/jira/browse/HIVE-6017


Repository: hive-git


Description
---

Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to 
Hive


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SignedInt128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SqlMathUtil.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestDecimal128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestSignedInt128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestSqlMathUtil.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestUnsignedInt128.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/16307/diff/


Testing
---


Thanks,

Eric Hanson

[jira] [Updated] (HIVE-6017) Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


 [ 
https://issues.apache.org/jira/browse/HIVE-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Hanson updated HIVE-6017:
--

Attachment: HIVE-6017.02.patch

Renamed new test classes to start with Test rather than end with Test per 
instructions from Brock Noland.

 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 ---

 Key: HIVE-6017
 URL: https://issues.apache.org/jira/browse/HIVE-6017
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Eric Hanson
 Attachments: HIVE-6017.01.patch, HIVE-6017.02.patch


 Contribute the Decimal128 high-performance decimal package developed by 
 Microsoft to Hive. This was originally written for Microsoft PolyBase by 
 Hideaki Kimura.
 This code is about 8X more efficient than Java BigDecimal for typical 
 operations. It uses a finite (128 bit) precision and can handle up to 
 decimal(38, X). It is also mutable so you can change the contents of an 
 existing object. This helps reduce the cost of new() and garbage collection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6021) Problem in GroupByOperator for handling distinct aggrgations

2013-12-16 Thread Sun Rui (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun Rui updated HIVE-6021:
--

Attachment: HIVE-6021.2.patch

attach the updated patch.

 Problem in GroupByOperator for handling distinct aggrgations
 

 Key: HIVE-6021
 URL: https://issues.apache.org/jira/browse/HIVE-6021
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.12.0
Reporter: Sun Rui
Assignee: Sun Rui
 Attachments: HIVE-6021.1.patch, HIVE-6021.2.patch


 Use the following test case with HIVE 0.12:
 {code:sql}
 create table src(key int, value string);
 load data local inpath 'src/data/files/kv1.txt' overwrite into table src;
 set hive.map.aggr=false; 
 select count(key),count(distinct value) from src group by key;
 {code}
 We will get an ArrayIndexOutOfBoundsException from GroupByOperator:
 {code}
 java.lang.RuntimeException: Error in configuring object
   at 
 org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
   at 
 org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
   at 
 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
   at 
 org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:485)
   at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
   at 
 org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
 Caused by: java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
   ... 5 more
 Caused by: java.lang.RuntimeException: Reduce operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:159)
   ... 10 more
 Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
   at 
 org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:281)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377)
   at 
 org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:152)
   ... 10 more
 {code}
 explain select count(key),count(distinct value) from src group by key;
 {code}
 STAGE PLANS:
   Stage: Stage-1
 Map Reduce
   Alias - Map Operator Tree:
 src 
   TableScan
 alias: src
 Select Operator
   expressions:
 expr: key
 type: int
 expr: value
 type: string
   outputColumnNames: key, value
   Reduce Output Operator
 key expressions:
   expr: key
   type: int
   expr: value
   type: string
 sort order: ++
 Map-reduce partition columns:
   expr: key
   type: int
 tag: -1
   Reduce Operator Tree:
 Group By Operator
   aggregations:
 expr: count(KEY._col0)   // The parameter causes this problem
^^^
 expr: count(DISTINCT KEY._col1:0._col0)
   bucketGroup: false
   keys:
 expr: KEY._col0
 type: int
   mode: complete
   outputColumnNames: _col0, _col1, _col2
   Select Operator
 expressions:
   expr: _col1
   type: bigint
   expr: _col2
   type: bigint
 outputColumnNames: _col0, _col1
 File Output Operator
   compressed: false
   GlobalTableId: 0
   table:
   input format: org.apache.hadoop.mapred.TextInputFormat
   output format: 
 org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
   Stage: Stage-0
 Fetch Operator
   limit: -1
 {code}
 The root cause is within GroupByOperator.initializeOp(). The method forgets 
 to handle the case:
 For a query has distinct aggregations, there is an aggregation function has a 
 parameter which is a groupby key column but not distinct key column.
 {code}
 if (unionExprEval != null) {
   String[] names = parameters.get(j).getExprString().split(\\.);
   // parameters of the form : KEY.colx:t.coly
   if (Utilities.ReduceField.KEY.name().equals(names[0])) {

Re: Review Request 16307: Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16307/
---

(Updated Dec. 17, 2013, 1:52 a.m.)


Review request for hive and Gunther Hagleitner.


Changes
---

Tweaked comment spacing and copyright.


Bugs: HIVE-6017
https://issues.apache.org/jira/browse/HIVE-6017


Repository: hive-git


Description
---

Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to 
Hive


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SignedInt128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SqlMathUtil.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestDecimal128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestSignedInt128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestSqlMathUtil.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestUnsignedInt128.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/16307/diff/


Testing
---


Thanks,

Eric Hanson

[jira] [Updated] (HIVE-6017) Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


 [ 
https://issues.apache.org/jira/browse/HIVE-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Hanson updated HIVE-6017:
--

Attachment: HIVE-6017.03.patch

 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 ---

 Key: HIVE-6017
 URL: https://issues.apache.org/jira/browse/HIVE-6017
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Eric Hanson
 Attachments: HIVE-6017.01.patch, HIVE-6017.02.patch, 
 HIVE-6017.03.patch


 Contribute the Decimal128 high-performance decimal package developed by 
 Microsoft to Hive. This was originally written for Microsoft PolyBase by 
 Hideaki Kimura.
 This code is about 8X more efficient than Java BigDecimal for typical 
 operations. It uses a finite (128 bit) precision and can handle up to 
 decimal(38, X). It is also mutable so you can change the contents of an 
 existing object. This helps reduce the cost of new() and garbage collection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-5521) Remove CommonRCFileInputFormat

2013-12-16 Thread Jitendra Nath Pandey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jitendra Nath Pandey updated HIVE-5521:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks to Ashutosh!

 Remove CommonRCFileInputFormat
 --

 Key: HIVE-5521
 URL: https://issues.apache.org/jira/browse/HIVE-5521
 Project: Hive
  Issue Type: Bug
  Components: File Formats, Vectorization
Affects Versions: 0.13.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
 Fix For: 0.13.0

 Attachments: HIVE-5521.patch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Review Request 16307: Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16307/
---

(Updated Dec. 17, 2013, 2:12 a.m.)


Review request for hive and Gunther Hagleitner.


Changes
---

removed trailing white space


Bugs: HIVE-6017
https://issues.apache.org/jira/browse/HIVE-6017


Repository: hive-git


Description
---

Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to 
Hive


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SignedInt128.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/SqlMathUtil.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestDecimal128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestSignedInt128.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestSqlMathUtil.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestUnsignedInt128.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/16307/diff/


Testing
---


Thanks,

Eric Hanson

[jira] [Updated] (HIVE-5521) Remove CommonRCFileInputFormat

2013-12-16 Thread Jitendra Nath Pandey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jitendra Nath Pandey updated HIVE-5521:
---

Fix Version/s: 0.13.0

 Remove CommonRCFileInputFormat
 --

 Key: HIVE-5521
 URL: https://issues.apache.org/jira/browse/HIVE-5521
 Project: Hive
  Issue Type: Bug
  Components: File Formats, Vectorization
Affects Versions: 0.13.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
 Fix For: 0.13.0

 Attachments: HIVE-5521.patch






--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (HIVE-6017) Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


 [ 
https://issues.apache.org/jira/browse/HIVE-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Hanson updated HIVE-6017:
--

Attachment: HIVE-6017.04.patch

remove trailing white space

 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 ---

 Key: HIVE-6017
 URL: https://issues.apache.org/jira/browse/HIVE-6017
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Eric Hanson
 Attachments: HIVE-6017.01.patch, HIVE-6017.02.patch, 
 HIVE-6017.03.patch, HIVE-6017.04.patch


 Contribute the Decimal128 high-performance decimal package developed by 
 Microsoft to Hive. This was originally written for Microsoft PolyBase by 
 Hideaki Kimura.
 This code is about 8X more efficient than Java BigDecimal for typical 
 operations. It uses a finite (128 bit) precision and can handle up to 
 decimal(38, X). It is also mutable so you can change the contents of an 
 existing object. This helps reduce the cost of new() and garbage collection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5230) Better error reporting by async threads in HiveServer2


[ 
https://issues.apache.org/jira/browse/HIVE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850007#comment-13850007
 ] 

Hive QA commented on HIVE-5230:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12618991/HIVE-5230.10.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4791 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
{noformat}

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/661/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/661/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12618991

 Better error reporting by async threads in HiveServer2
 --

 Key: HIVE-5230
 URL: https://issues.apache.org/jira/browse/HIVE-5230
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0, 0.13.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
 Fix For: 0.13.0

 Attachments: HIVE-5230.1.patch, HIVE-5230.1.patch, 
 HIVE-5230.10.patch, HIVE-5230.2.patch, HIVE-5230.3.patch, HIVE-5230.4.patch, 
 HIVE-5230.6.patch, HIVE-5230.7.patch, HIVE-5230.8.patch, HIVE-5230.9.patch


 [HIVE-4617|https://issues.apache.org/jira/browse/HIVE-4617] provides support 
 for async execution in HS2. When a background thread gets an error, currently 
 the client can only poll for the operation state and also the error with its 
 stacktrace is logged. However, it will be useful to provide a richer error 
 response like thrift API does with TStatus (which is constructed while 
 building a Thrift response object). 



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-6017) Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive


[ 
https://issues.apache.org/jira/browse/HIVE-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850047#comment-13850047
 ] 

Hive QA commented on HIVE-6017:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12619021/HIVE-6017.04.patch

{color:green}SUCCESS:{color} +1 4840 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/662/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/662/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12619021

 Contribute Decimal128 high-performance decimal(p, s) package from Microsoft 
 to Hive
 ---

 Key: HIVE-6017
 URL: https://issues.apache.org/jira/browse/HIVE-6017
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Eric Hanson
 Attachments: HIVE-6017.01.patch, HIVE-6017.02.patch, 
 HIVE-6017.03.patch, HIVE-6017.04.patch


 Contribute the Decimal128 high-performance decimal package developed by 
 Microsoft to Hive. This was originally written for Microsoft PolyBase by 
 Hideaki Kimura.
 This code is about 8X more efficient than Java BigDecimal for typical 
 operations. It uses a finite (128 bit) precision and can handle up to 
 decimal(38, X). It is also mutable so you can change the contents of an 
 existing object. This helps reduce the cost of new() and garbage collection.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Hive-trunk-hadoop2 - Build # 606 - Still Failing

Changes for Build #571
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #572
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #573
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #574
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #575
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #576

Changes for Build #577

Changes for Build #578

Changes for Build #579
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #580
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #581
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #582
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #583
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #584
[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)

[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #585
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)


Changes for Build #586
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #587
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore direct sql doesn't work with oracle (Sergey 
Shelukhin via Ashutosh Chauhan)


Changes for Build #588

Changes for Build #589

Changes for Build #590
[brock] HIVE-5981 - Add hive-unit back to itests pom (Brock Noland reviewed by 
Prasad)


Changes for Build #591
[xuefu] HIVE-5872: Make UDAFs such as GenericUDAFSum report

[jira] [Updated] (HIVE-1466) Add NULL DEFINED AS to ROW FORMAT specification

2013-12-16 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-1466:


Issue Type: New Feature  (was: Improvement)

 Add NULL DEFINED AS to ROW FORMAT specification
 ---

 Key: HIVE-1466
 URL: https://issues.apache.org/jira/browse/HIVE-1466
 Project: Hive
  Issue Type: New Feature
Reporter: Adam Kramer
Assignee: Prasad Mujumdar
 Attachments: HIVE-1466.1.patch, HIVE-1466.2.patch


 NULL values are passed to transformers as a literal backslash and a literal 
 N. NULL values are saved when INSERT OVERWRITing LOCAL DIRECTORies as NULL. 
 This is inconsistent.
 The ROW FORMAT specification of tables should be able to specify the manner 
 in which a null character is represented. ROW FORMAT NULL DEFINED AS '\N' or 
 '\003' or whatever should apply to all instances of table export and saving.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Hive-trunk-h0.21 - Build # 2507 - Still Failing

Changes for Build #2472
[navis] HIVE-4518 : Should be removed files (OptrStatsGroupByHook, etc.)

[navis] HIVE-5839 : BytesRefArrayWritable compareTo violates contract (Xuefu 
Zhang via Navis)

[navis] HIVE-4518 : Missing file (HiveFatalException)

[navis] HIVE-4518 : Counter Strike: Operation Operator (Gunther Hagleitner and 
Jason Dere via Navis)


Changes for Build #2473
[brock] HIVE-4741 - Add Hive config API to modify the restrict list (Prasad 
Mujumdar, Navis via Brock Noland)


Changes for Build #2474
[navis] HIVE-5827 : Incorrect location of logs for failed tests (Vikram Dixit K 
and Szehon Ho via Navis)

[thejas] HIVE-4485 : beeline prints null as empty strings (Thejas Nair reviewed 
by Ashutosh Chauhan)

[brock] HIVE-5704 - A couple of generic UDFs are not in the right 
folder/package (Xuefu Zhang via Brock Noland)

[brock] HIVE-5706 - Move a few numeric UDFs to generic implementations (Xuefu 
Zhang via Brock Noland)

[hashutosh] HIVE-5817 : column name to index mapping in VectorizationContext is 
broken (Remus Rusanu, Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5876 : Split elimination in ORC breaks for partitioned tables 
(Prasanth J via Ashutosh Chauhan)

[hashutosh] HIVE-5886 : [Refactor] Remove unused class JobCloseFeedback 
(Ashutosh Chauhan via Thejas Nair)

[brock] HIVE-5894 - Fix minor PTest2 issues (Brock Noland)


Changes for Build #2475
[brock] HIVE-5755 - Fix hadoop2 execution environment Milestone 1 (Vikram Dixit 
K via Brock Noland)


Changes for Build #2476
[xuefu] HIVE-5893: hive-schema-0.13.0.mysql.sql contains reference to 
nonexistent column (Carl via Xuefu)

[xuefu] HIVE-5684: Serde support for char (Jason via Xuefu)


Changes for Build #2477

Changes for Build #2478

Changes for Build #2479

Changes for Build #2480
[brock] HIVE-5441 - Async query execution doesn't return resultset status 
(Prasad Mujumdar via Thejas M Nair)

[brock] HIVE-5880 - Rename HCatalog HBase Storage Handler artifact id (Brock 
Noland reviewed by Prasad Mujumdar)


Changes for Build #2481

Changes for Build #2482
[ehans] HIVE-5581: Implement vectorized year/month/day... etc. for string 
arguments (Teddy Choi via Eric Hanson)


Changes for Build #2483
[rhbutani] HIVE-5898 Make fetching of column statistics configurable (Prasanth 
Jayachandran via Harish Butani)


Changes for Build #2484
[brock] HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks 
packaging (Xuefu Zhang via Brock Noland)


Changes for Build #2485
[xuefu] HIVE-5866: Hive divide operator generates wrong results in certain 
cases (reviewed by Prasad)

[ehans] HIVE-5877: Implement vectorized support for IN as boolean-valued 
expression (Eric Hanson)


Changes for Build #2486
[ehans] HIVE-5895: vectorization handles division by zero differently from 
normal execution (Sergey Shelukhin via Eric Hanson)

[hashutosh] HIVE-5938 : Remove apache.mina dependency for test (Navis via 
Ashutosh Chauhan)

[xuefu] HIVE-5912: Show partition command doesn't support db.table (Yu Zhao via 
Xuefu)

[brock] HIVE-5906 - TestGenericUDFPower should use delta to compare doubles 
(Szehon Ho via Brock Noland)

[brock] HIVE-5855 - Add deprecated methods back to ColumnProjectionUtils (Brock 
Noland reviewed by Navis)

[brock] HIVE-5915 - Shade Kryo dependency (Brock Noland reviewed by Ashutosh 
Chauhan)


Changes for Build #2487
[hashutosh] HIVE-5916 : No need to aggregate statistics collected via counter 
mechanism (Ashutosh Chauhan via Navis)

[xuefu] HIVE-5947: Fix test failure in decimal_udf.q (reviewed by Brock)

[thejas] HIVE-5550 : Import fails for tables created with default text, 
sequence and orc file formats using HCatalog API (Sushanth Sowmyan via Thejas 
Nair)


Changes for Build #2488
[hashutosh] HIVE-5935 : hive.query.string is not provided to FetchTask (Navis 
via Ashutosh Chauhan)

[navis] HIVE-3455 : ANSI CORR(X,Y) is incorrect (Maxim Bolotin via Navis)

[hashutosh] HIVE-5921 : Better heuristics for worst case statistics estimates 
for join, limit and filter operator (Prasanth J via Harish Butani)

[rhbutani] HIVE-5899 NPE during explain extended with char/varchar columns 
(Jason Dere via Harish Butani)


Changes for Build #2489
[xuefu] HIVE-3181: getDatabaseMajor/Minor version does not return values 
(Szehon via Xuefu, reviewed by Navis)

[brock] HIVE-5641 - BeeLineOpts ignores Throwable (Brock Noland reviewed by 
Prasad and Thejas)

[hashutosh] HIVE-5909 : locate and instr throw 
java.nio.BufferUnderflowException when empty string as substring (Navis via 
Ashutosh Chauhan)

[hashutosh] HIVE-5686 : partition column type validation doesn't quite work for 
dates (Sergey Shelukhin via Ashutosh Chauhan)

[hashutosh] HIVE-5887 : metastore direct sql doesn't work with oracle (Sergey 
Shelukhin via Ashutosh Chauhan)


Changes for Build #2490

Changes for Build #2491

Changes for Build #2492
[brock] HIVE-5981 - Add hive-unit back to itests pom (Brock Noland reviewed by 
Prasad)


Changes for Build #2493
[xuefu] HIVE-5872:

[jira] [Commented] (HIVE-1466) Add NULL DEFINED AS to ROW FORMAT specification

2013-12-16 Thread Thejas M Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850063#comment-13850063
 ] 

Thejas M Nair commented on HIVE-1466:
-

Prasad, can you please add a release note to the jira and create a followup 
jira for inclusion in wiki (Lefty or someone else might be able to help with 
incorporation in wiki) or update the wiki page itself directly ?
We should try to ensure that all new features get documented. I think the best 
way to do that is to ensure that documentation is available before feature is 
committed.


 Add NULL DEFINED AS to ROW FORMAT specification
 ---

 Key: HIVE-1466
 URL: https://issues.apache.org/jira/browse/HIVE-1466
 Project: Hive
  Issue Type: New Feature
Reporter: Adam Kramer
Assignee: Prasad Mujumdar
 Attachments: HIVE-1466.1.patch, HIVE-1466.2.patch


 NULL values are passed to transformers as a literal backslash and a literal 
 N. NULL values are saved when INSERT OVERWRITing LOCAL DIRECTORies as NULL. 
 This is inconsistent.
 The ROW FORMAT specification of tables should be able to specify the manner 
 in which a null character is represented. ROW FORMAT NULL DEFINED AS '\N' or 
 '\003' or whatever should apply to all instances of table export and saving.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HIVE-5829) Rewrite Trim and Pad UDFs based on GenericUDF