[jira] [Commented] (HIVE-13566) Auto-gather column stats - phase 1
[ https://issues.apache.org/jira/browse/HIVE-13566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307388#comment-16307388 ] Lefty Leverenz commented on HIVE-13566: --- Good docs, thanks Zoltan. > Auto-gather column stats - phase 1 > -- > > Key: HIVE-13566 > URL: https://issues.apache.org/jira/browse/HIVE-13566 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-13566.01.patch, HIVE-13566.02.patch, > HIVE-13566.03.patch > > > This jira adds code and tests for auto-gather column stats. Golden file > update will be done in phase 2 - HIVE-11160 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18230) create plan like plan, and replace plan commands for easy modification
[ https://issues.apache.org/jira/browse/HIVE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307379#comment-16307379 ] Lefty Leverenz commented on HIVE-18230: --- Doc note: This needs to be documented in the Explain wikidoc, and perhaps also with the new workload management feature. ACTIVE needs to be listed in the DDL doc as a non-reserved keyword, along with WORKLOAD and MANAGEMENT (created by HIVE-18203). * [Explain Plan | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Explain] * [DDL -- Non-reserved Keywords| https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Non-reservedKeywords] Added a TODOC3.0 label. (Please add your own TODOC labels and doc notes in the future.) > create plan like plan, and replace plan commands for easy modification > -- > > Key: HIVE-18230 > URL: https://issues.apache.org/jira/browse/HIVE-18230 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18230.01.patch, HIVE-18230.only.nogen.patch, > HIVE-18230.patch > > > Given that the plan already on the cluster cannot be altered, it would be > helpful to have create plan like plan, and replace plan commands that would > make a copy to be modified, and then rename+apply the copy in place of an > existing plan, and rename the existing active plan with a versioned name or > drop it altogether. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18230) create plan like plan, and replace plan commands for easy modification
[ https://issues.apache.org/jira/browse/HIVE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-18230: -- Labels: TODOC3.0 (was: ) > create plan like plan, and replace plan commands for easy modification > -- > > Key: HIVE-18230 > URL: https://issues.apache.org/jira/browse/HIVE-18230 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18230.01.patch, HIVE-18230.only.nogen.patch, > HIVE-18230.patch > > > Given that the plan already on the cluster cannot be altered, it would be > helpful to have create plan like plan, and replace plan commands that would > make a copy to be modified, and then rename+apply the copy in place of an > existing plan, and rename the existing active plan with a versioned name or > drop it altogether. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18203) change the way WM is enabled and allow dropping the last resource plan
[ https://issues.apache.org/jira/browse/HIVE-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307377#comment-16307377 ] Lefty Leverenz commented on HIVE-18203: --- Doc note: This needs to be documented with the rest of the workload management feature, including the new keywords WORKLOAD and MANAGEMENT which should be added in the DDL doc. They are reserved keywords in this patch but HIVE-18230 makes them non-reserved. * [DDL -- Keywords | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Keywords,Non-reservedKeywordsandReservedKeywords] Added a TODOC3.0 label. (Please add your own TODOC labels and doc notes in the future.) > change the way WM is enabled and allow dropping the last resource plan > -- > > Key: HIVE-18203 > URL: https://issues.apache.org/jira/browse/HIVE-18203 > Project: Hive > Issue Type: Sub-task >Reporter: Aswathy Chellammal Sreekumar >Assignee: Sergey Shelukhin > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18203.01.patch, HIVE-18203.02.patch, > HIVE-18203.03.patch, HIVE-18203.04.patch, HIVE-18203.patch > > > Currently it's impossible to drop the last active resource plan even if WM is > disabled. It should be possible to deactivate the last resource plan AND > disable WM in the same action. Activating a resource plan should enable WM in > this case. > This should interact with the WM queue config in a sensible manner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18203) change the way WM is enabled and allow dropping the last resource plan
[ https://issues.apache.org/jira/browse/HIVE-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-18203: -- Labels: TODOC3.0 (was: ) > change the way WM is enabled and allow dropping the last resource plan > -- > > Key: HIVE-18203 > URL: https://issues.apache.org/jira/browse/HIVE-18203 > Project: Hive > Issue Type: Sub-task >Reporter: Aswathy Chellammal Sreekumar >Assignee: Sergey Shelukhin > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18203.01.patch, HIVE-18203.02.patch, > HIVE-18203.03.patch, HIVE-18203.04.patch, HIVE-18203.patch > > > Currently it's impossible to drop the last active resource plan even if WM is > disabled. It should be possible to deactivate the last resource plan AND > disable WM in the same action. Activating a resource plan should enable WM in > this case. > This should interact with the WM queue config in a sensible manner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18224) Introduce interface above driver
[ https://issues.apache.org/jira/browse/HIVE-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307375#comment-16307375 ] Lefty Leverenz commented on HIVE-18224: --- Does this need to be documented in the wiki? > Introduce interface above driver > > > Key: HIVE-18224 > URL: https://issues.apache.org/jira/browse/HIVE-18224 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Fix For: 3.0.0 > > Attachments: HIVE-18224.01.patch, HIVE-18224.02.patch, > HIVE-18224.03.patch, HIVE-18224.04.patch > > > Add an interface above driver; and use it outside of ql. > The goal is to enable the overlaying of the Driver with some strategy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18095) add a unmanaged flag to triggers (applies to container based sessions)
[ https://issues.apache.org/jira/browse/HIVE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307372#comment-16307372 ] Lefty Leverenz commented on HIVE-18095: --- Doc note: This adds the non-reserved keyword UNMANAGED, which needs to be documented in the wiki. * [Non-reserved Keywords | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Non-reservedKeywords] Added a TODOC3.0 label. (Please add your own TODOC labels and doc notes in the future.) > add a unmanaged flag to triggers (applies to container based sessions) > -- > > Key: HIVE-18095 > URL: https://issues.apache.org/jira/browse/HIVE-18095 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18095.01.patch, HIVE-18095.02.patch, > HIVE-18095.nogen.patch, HIVE-18095.patch > > > cc [~prasanth_j] > It should be impossible to attach global triggers for pools. Setting global > flag should probably automatically remove attachments to pools. > Global triggers would only support actions that Tez supports (for simplicity; > also, for now, move doesn't make a lot of sense because the trigger would > apply again after the move). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18095) add a unmanaged flag to triggers (applies to container based sessions)
[ https://issues.apache.org/jira/browse/HIVE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-18095: -- Labels: TODOC3.0 (was: ) > add a unmanaged flag to triggers (applies to container based sessions) > -- > > Key: HIVE-18095 > URL: https://issues.apache.org/jira/browse/HIVE-18095 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18095.01.patch, HIVE-18095.02.patch, > HIVE-18095.nogen.patch, HIVE-18095.patch > > > cc [~prasanth_j] > It should be impossible to attach global triggers for pools. Setting global > flag should probably automatically remove attachments to pools. > Global triggers would only support actions that Tez supports (for simplicity; > also, for now, move doesn't make a lot of sense because the trigger would > apply again after the move). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-1662) Add file pruning into Hive.
[
https://issues.apache.org/jira/browse/HIVE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307161#comment-16307161
]
Hive QA commented on HIVE-1662:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12653796/HIVE-1662.18.patch.txt
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8391/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8391/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8391/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-12-31 09:33:31.708
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8391/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-12-31 09:33:31.710
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 035eca3 HIVE-18331 : Renew the Kerberos ticket used by Druid
Query runner (Slim Bouguerra via Sergey Shelukhin)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 035eca3 HIVE-18331 : Renew the Kerberos ticket used by Druid
Query runner (Slim Bouguerra via Sergey Shelukhin)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-12-31 09:33:36.750
+ rm -rf ../yetus
+ mkdir ../yetus
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8391/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:631
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
with conflicts.
error: itests/qtest/testconfiguration.properties: does not exist in index
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/Context.java:37
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/Context.java' with
conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java:1509
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java' with
conflicts.
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java:515
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java' with
conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:2170
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java:36
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java' with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java:39
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java' with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java:28
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java:62
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java' with
conflicts.
error: patch failed:
[jira] [Commented] (HIVE-1662) Add file pruning into Hive.
[ https://issues.apache.org/jira/browse/HIVE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307158#comment-16307158 ] Alexander Libov commented on HIVE-1662: --- Anyone working on this? It will greatly increase usability for external tables where the data is not generated by Hive. > Add file pruning into Hive. > --- > > Key: HIVE-1662 > URL: https://issues.apache.org/jira/browse/HIVE-1662 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Navis > Attachments: HIVE-1662.10.patch.txt, HIVE-1662.11.patch.txt, > HIVE-1662.12.patch.txt, HIVE-1662.13.patch.txt, HIVE-1662.14.patch.txt, > HIVE-1662.15.patch.txt, HIVE-1662.16.patch.txt, HIVE-1662.17.patch.txt, > HIVE-1662.18.patch.txt, HIVE-1662.8.patch.txt, HIVE-1662.9.patch.txt, > HIVE-1662.D8391.1.patch, HIVE-1662.D8391.2.patch, HIVE-1662.D8391.3.patch, > HIVE-1662.D8391.4.patch, HIVE-1662.D8391.5.patch, HIVE-1662.D8391.6.patch, > HIVE-1662.D8391.7.patch > > > now hive support filename virtual column. > if a file name filter presents in a query, hive should be able to only add > files which passed the filter to input paths. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
