[jira] [Commented] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585027#comment-16585027
]
Hive QA commented on HIVE-17979:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 1 new + 136 unchanged - 0
fixed = 137 total (was 136) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13336/dev-support/hive-personality.sh
|
| git revision | master / 0f772ed |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13336/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13336/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13336/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch, HIVE-17979.4.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20417) Hive converts strings to doubles in a strange manner
[
https://issues.apache.org/jira/browse/HIVE-20417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585025#comment-16585025
]
Ashutosh Chauhan commented on HIVE-20417:
-
Couldn't find anything in standard so I checked on few other DBs. Surprisingly,
mysql and oracle has same behavior as Hive for {{c1.i + c2.i}} However they
throw exception for {{c1.i + ' ' + c2.i}} Postgres throws for both. Sql server
does string concat in both cases. Personally, I liked sql server behavior and
expected that behavior. However, since standard doesnt say anything on this
topic (maybe it did but I couldnt find it) and few other DBs do exhibit same
behavior, I think there is no need to change anything for Hive. For second
case, throwing exception is an option but historically Hive has chosen to
return null instead of exception in such ambiguous cases.
So, I will say this is "working as designed".
> Hive converts strings to doubles in a strange manner
>
>
> Key: HIVE-20417
> URL: https://issues.apache.org/jira/browse/HIVE-20417
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Matt McCline
>Priority: Major
>
> This is not on master, but on a close enough version, so it needs to be dbl
> checked... running with vectorization on:
> {noformat}
> POSTHOOK: query: create table cross_numbers(i string)
> POSTHOOK: query: insert into table cross_numbers select key from src limit 20
> POSTHOOK: query: select c1.i, c2.i, c1.i + c2.i, c1.i + '' + c2.i from
> cross_numbers c1 cross join cross_numbers c2
> 401 265 666.0 NULL
> 401 255 656.0 NULL
> 401 165 566.0 NULL
> {noformat}
> Why are strings added as numbers, and string + '' + string converted to null
> is not clear..
> [~ashutoshc] is plus supposed to work for string?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[ https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20420: -- Attachment: HIVE-20420.3.patch > Provide a fallback authorizer when no other authorizer is in use > > > Key: HIVE-20420 > URL: https://issues.apache.org/jira/browse/HIVE-20420 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20420.1.patch, HIVE-20420.2.patch, > HIVE-20420.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585009#comment-16585009
]
Ashutosh Chauhan commented on HIVE-20421:
-
+1
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-17979:
Status: Patch Available (was: Open)
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch, HIVE-17979.4.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-17979:
Attachment: HIVE-17979.4.patch
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch, HIVE-17979.4.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-17979:
Status: Open (was: Patch Available)
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20366) TPC-DS query78 stats estimates are off for is null filter
[
https://issues.apache.org/jira/browse/HIVE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585006#comment-16585006
]
Hive QA commented on HIVE-20366:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936172/HIVE-20366.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13335/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13335/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13335/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12936172/HIVE-20366.4.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936172 - PreCommit-HIVE-Build
> TPC-DS query78 stats estimates are off for is null filter
> -
>
> Key: HIVE-20366
> URL: https://issues.apache.org/jira/browse/HIVE-20366
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20366.1.patch, HIVE-20366.2.patch,
> HIVE-20366.3.patch, HIVE-20366.4.patch
>
>
> In Query 78, there is Left outer join between fact table combos: stores_sales
> LOJ store_returns, catalog_sales LOJ catalog_returns and web_sales LOJ
> web_returns. Each of these joins estimates only a single row and the result
> is BROADCAST and causes hash table memory errors
> {code}
> Reducer 12 |
> | Execution mode: vectorized, llap |
> | Reduce Operator Tree: |
> ++
> | Explain |
> ++
> | Map Join Operator|
> | condition map: |
> | Left Outer Join 0 to 1|
> | keys: |
> | 0 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | 1 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | outputColumnNames: _col0, _col1, _col3, _col4, _col5,
> _col6, _col8 |
> | input vertices:|
> | 1 Map 14 |
> | Statistics: Num rows: 10282477384 Data size: 534184867432
> Basic stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator|
> | predicate: _col8 is null (type: boolean) |
> | * Statistics: Num rows: 1* Data size: 52 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585005#comment-16585005
]
Hive QA commented on HIVE-17979:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936170/HIVE-17979.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13334/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13334/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13334/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12936170/HIVE-17979.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936170 - PreCommit-HIVE-Build
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20366) TPC-DS query78 stats estimates are off for is null filter
[
https://issues.apache.org/jira/browse/HIVE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585004#comment-16585004
]
Ashutosh Chauhan commented on HIVE-20366:
-
+1
> TPC-DS query78 stats estimates are off for is null filter
> -
>
> Key: HIVE-20366
> URL: https://issues.apache.org/jira/browse/HIVE-20366
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20366.1.patch, HIVE-20366.2.patch,
> HIVE-20366.3.patch, HIVE-20366.4.patch
>
>
> In Query 78, there is Left outer join between fact table combos: stores_sales
> LOJ store_returns, catalog_sales LOJ catalog_returns and web_sales LOJ
> web_returns. Each of these joins estimates only a single row and the result
> is BROADCAST and causes hash table memory errors
> {code}
> Reducer 12 |
> | Execution mode: vectorized, llap |
> | Reduce Operator Tree: |
> ++
> | Explain |
> ++
> | Map Join Operator|
> | condition map: |
> | Left Outer Join 0 to 1|
> | keys: |
> | 0 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | 1 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | outputColumnNames: _col0, _col1, _col3, _col4, _col5,
> _col6, _col8 |
> | input vertices:|
> | 1 Map 14 |
> | Statistics: Num rows: 10282477384 Data size: 534184867432
> Basic stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator|
> | predicate: _col8 is null (type: boolean) |
> | * Statistics: Num rows: 1* Data size: 52 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20366) TPC-DS query78 stats estimates are off for is null filter
[
https://issues.apache.org/jira/browse/HIVE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584980#comment-16584980
]
Hive QA commented on HIVE-20366:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936172/HIVE-20366.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14885 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1]
(batchId=194)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/1/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936172 - PreCommit-HIVE-Build
> TPC-DS query78 stats estimates are off for is null filter
> -
>
> Key: HIVE-20366
> URL: https://issues.apache.org/jira/browse/HIVE-20366
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20366.1.patch, HIVE-20366.2.patch,
> HIVE-20366.3.patch, HIVE-20366.4.patch
>
>
> In Query 78, there is Left outer join between fact table combos: stores_sales
> LOJ store_returns, catalog_sales LOJ catalog_returns and web_sales LOJ
> web_returns. Each of these joins estimates only a single row and the result
> is BROADCAST and causes hash table memory errors
> {code}
> Reducer 12 |
> | Execution mode: vectorized, llap |
> | Reduce Operator Tree: |
> ++
> | Explain |
> ++
> | Map Join Operator|
> | condition map: |
> | Left Outer Join 0 to 1|
> | keys: |
> | 0 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | 1 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | outputColumnNames: _col0, _col1, _col3, _col4, _col5,
> _col6, _col8 |
> | input vertices:|
> | 1 Map 14 |
> | Statistics: Num rows: 10282477384 Data size: 534184867432
> Basic stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator|
> | predicate: _col8 is null (type: boolean) |
> | * Statistics: Num rows: 1* Data size: 52 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20366) TPC-DS query78 stats estimates are off for is null filter
[
https://issues.apache.org/jira/browse/HIVE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584975#comment-16584975
]
Hive QA commented on HIVE-20366:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
17s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 2 new + 23 unchanged - 0 fixed
= 25 total (was 23) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-1/dev-support/hive-personality.sh
|
| git revision | master / 0f772ed |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-1/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-1/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TPC-DS query78 stats estimates are off for is null filter
> -
>
> Key: HIVE-20366
> URL: https://issues.apache.org/jira/browse/HIVE-20366
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20366.1.patch, HIVE-20366.2.patch,
> HIVE-20366.3.patch, HIVE-20366.4.patch
>
>
> In Query 78, there is Left outer join between fact table combos: stores_sales
> LOJ store_returns, catalog_sales LOJ catalog_returns and web_sales LOJ
> web_returns. Each of these joins estimates only a single row and the result
> is BROADCAST and causes hash table memory errors
> {code}
> Reducer 12 |
> | Execution mode: vectorized, llap |
> | Reduce Operator Tree: |
> ++
> | Explain |
> ++
> | Map Join Operator|
> | condition map: |
> | Left Outer Join 0 to 1|
> | keys: |
> | 0 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | 1 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
>
[jira] [Updated] (HIVE-20366) TPC-DS query78 stats estimates are off for is null filter
[
https://issues.apache.org/jira/browse/HIVE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20366:
---
Status: Patch Available (was: Open)
> TPC-DS query78 stats estimates are off for is null filter
> -
>
> Key: HIVE-20366
> URL: https://issues.apache.org/jira/browse/HIVE-20366
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20366.1.patch, HIVE-20366.2.patch,
> HIVE-20366.3.patch, HIVE-20366.4.patch
>
>
> In Query 78, there is Left outer join between fact table combos: stores_sales
> LOJ store_returns, catalog_sales LOJ catalog_returns and web_sales LOJ
> web_returns. Each of these joins estimates only a single row and the result
> is BROADCAST and causes hash table memory errors
> {code}
> Reducer 12 |
> | Execution mode: vectorized, llap |
> | Reduce Operator Tree: |
> ++
> | Explain |
> ++
> | Map Join Operator|
> | condition map: |
> | Left Outer Join 0 to 1|
> | keys: |
> | 0 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | 1 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | outputColumnNames: _col0, _col1, _col3, _col4, _col5,
> _col6, _col8 |
> | input vertices:|
> | 1 Map 14 |
> | Statistics: Num rows: 10282477384 Data size: 534184867432
> Basic stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator|
> | predicate: _col8 is null (type: boolean) |
> | * Statistics: Num rows: 1* Data size: 52 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20366) TPC-DS query78 stats estimates are off for is null filter
[
https://issues.apache.org/jira/browse/HIVE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20366:
---
Attachment: HIVE-20366.4.patch
> TPC-DS query78 stats estimates are off for is null filter
> -
>
> Key: HIVE-20366
> URL: https://issues.apache.org/jira/browse/HIVE-20366
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20366.1.patch, HIVE-20366.2.patch,
> HIVE-20366.3.patch, HIVE-20366.4.patch
>
>
> In Query 78, there is Left outer join between fact table combos: stores_sales
> LOJ store_returns, catalog_sales LOJ catalog_returns and web_sales LOJ
> web_returns. Each of these joins estimates only a single row and the result
> is BROADCAST and causes hash table memory errors
> {code}
> Reducer 12 |
> | Execution mode: vectorized, llap |
> | Reduce Operator Tree: |
> ++
> | Explain |
> ++
> | Map Join Operator|
> | condition map: |
> | Left Outer Join 0 to 1|
> | keys: |
> | 0 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | 1 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | outputColumnNames: _col0, _col1, _col3, _col4, _col5,
> _col6, _col8 |
> | input vertices:|
> | 1 Map 14 |
> | Statistics: Num rows: 10282477384 Data size: 534184867432
> Basic stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator|
> | predicate: _col8 is null (type: boolean) |
> | * Statistics: Num rows: 1* Data size: 52 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20366) TPC-DS query78 stats estimates are off for is null filter
[
https://issues.apache.org/jira/browse/HIVE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20366:
---
Status: Open (was: Patch Available)
> TPC-DS query78 stats estimates are off for is null filter
> -
>
> Key: HIVE-20366
> URL: https://issues.apache.org/jira/browse/HIVE-20366
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20366.1.patch, HIVE-20366.2.patch,
> HIVE-20366.3.patch, HIVE-20366.4.patch
>
>
> In Query 78, there is Left outer join between fact table combos: stores_sales
> LOJ store_returns, catalog_sales LOJ catalog_returns and web_sales LOJ
> web_returns. Each of these joins estimates only a single row and the result
> is BROADCAST and causes hash table memory errors
> {code}
> Reducer 12 |
> | Execution mode: vectorized, llap |
> | Reduce Operator Tree: |
> ++
> | Explain |
> ++
> | Map Join Operator|
> | condition map: |
> | Left Outer Join 0 to 1|
> | keys: |
> | 0 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | 1 KEY.reducesinkkey0 (type: bigint), KEY.reducesinkkey1
> (type: bigint) |
> | outputColumnNames: _col0, _col1, _col3, _col4, _col5,
> _col6, _col8 |
> | input vertices:|
> | 1 Map 14 |
> | Statistics: Num rows: 10282477384 Data size: 534184867432
> Basic stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator|
> | predicate: _col8 is null (type: boolean) |
> | * Statistics: Num rows: 1* Data size: 52 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19674) Group by Decimal Constants push down to Druid tables.
[
https://issues.apache.org/jira/browse/HIVE-19674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584963#comment-16584963
]
Hive QA commented on HIVE-19674:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12935920/HIVE-19674.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 14885 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constGby] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin4] (batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stat_estimate_drill]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp] (batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_null_projection]
(batchId=10)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_partition_pruning]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_8]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_2]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_null_agg]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_sort_11]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_null_projection]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_partition_pruning]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=187)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=187)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in]
(batchId=138)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin]
(batchId=141)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_null_agg]
(batchId=141)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query44]
(batchId=266)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query44]
(batchId=264)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13331/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13331/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13331/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 26 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12935920 - PreCommit-HIVE-Build
> Group by Decimal Constants push down to Druid tables.
> -
>
> Key: HIVE-19674
> URL: https://issues.apache.org/jira/browse/HIVE-19674
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19674.1.patch, HIVE-19674.2.patch
>
>
> Queries like following gets generated by Tableau.
> {code}
> SELECT SUM(`ssb_druid_100`.`lo_revenue`) AS `sum_lo_revenue_ok`
> FROM `druid_ssb`.`ssb_druid_100` `ssb_druid_100`
> GROUP BY 1.1001;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19674) Group by Decimal Constants push down to Druid tables.
[
https://issues.apache.org/jira/browse/HIVE-19674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584959#comment-16584959
]
Hive QA commented on HIVE-19674:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} ql: The patch generated 0 new + 1 unchanged - 4
fixed = 1 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13331/dev-support/hive-personality.sh
|
| git revision | master / 0f772ed |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13331/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Group by Decimal Constants push down to Druid tables.
> -
>
> Key: HIVE-19674
> URL: https://issues.apache.org/jira/browse/HIVE-19674
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19674.1.patch, HIVE-19674.2.patch
>
>
> Queries like following gets generated by Tableau.
> {code}
> SELECT SUM(`ssb_druid_100`.`lo_revenue`) AS `sum_lo_revenue_ok`
> FROM `druid_ssb`.`ssb_druid_100` `ssb_druid_100`
> GROUP BY 1.1001;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584950#comment-16584950
]
Hive QA commented on HIVE-17979:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936170/HIVE-17979.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14885 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1]
(batchId=194)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13330/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13330/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13330/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936170 - PreCommit-HIVE-Build
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19674) Group by Decimal Constants push down to Druid tables.
[
https://issues.apache.org/jira/browse/HIVE-19674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584946#comment-16584946
]
Vineet Garg commented on HIVE-19674:
bq. Wonder if its possible to avoid this call altogether
getPulledUpPredicates() is used to figure out if any of the input ref
expressions are constant. I am not sure what else to use to figure it out. I
notice calcite rules like AggregateProjectPullUpConstantsRule use the same call.
bq. can we make that condition even more tighter?
Since we want to replace all {{group by }} I am not sure how
can this call be made more tight.
[~jcamachorodriguez] any suggestions?
> Group by Decimal Constants push down to Druid tables.
> -
>
> Key: HIVE-19674
> URL: https://issues.apache.org/jira/browse/HIVE-19674
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19674.1.patch, HIVE-19674.2.patch
>
>
> Queries like following gets generated by Tableau.
> {code}
> SELECT SUM(`ssb_druid_100`.`lo_revenue`) AS `sum_lo_revenue_ok`
> FROM `druid_ssb`.`ssb_druid_100` `ssb_druid_100`
> GROUP BY 1.1001;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584941#comment-16584941
]
Hive QA commented on HIVE-17979:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 1 new + 136 unchanged - 0
fixed = 137 total (was 136) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13330/dev-support/hive-personality.sh
|
| git revision | master / 0f772ed |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13330/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13330/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13330/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-17979:
Attachment: HIVE-17979.3.patch
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-17979:
Status: Patch Available (was: Open)
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch,
> HIVE-17979.3.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-17979:
Status: Open (was: Patch Available)
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584924#comment-16584924
]
Hive QA commented on HIVE-17979:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12896290/HIVE-17979.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13329/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13329/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13329/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12896290/HIVE-17979.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12896290 - PreCommit-HIVE-Build
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[
https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584900#comment-16584900
]
Hive QA commented on HIVE-20420:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936164/HIVE-20420.2.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14893 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_coalesce]
(batchId=179)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13328/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13328/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13328/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936164 - PreCommit-HIVE-Build
> Provide a fallback authorizer when no other authorizer is in use
>
>
> Key: HIVE-20420
> URL: https://issues.apache.org/jira/browse/HIVE-20420
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20420.1.patch, HIVE-20420.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18453) ACID: Add "CREATE TRANSACTIONAL TABLE" syntax to unify ACID ORC & Parquet support
[
https://issues.apache.org/jira/browse/HIVE-18453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584882#comment-16584882
]
Igor Kryvenko commented on HIVE-18453:
--
[~ekoifman] Hi Eugene. I've updated my patch. I've removed {{temporary=true}}.
Also, built hive from source, and run {{create_transactional.q}} file and
everything works fine.
Also, can you clarify what did you mean "_so that this method acts as if
CREATE_TABLES_AS_ACID and HIVE_CREATE_TABLES_AS_INSERT_ONLY are both true._"?
Thanks, Igor.
> ACID: Add "CREATE TRANSACTIONAL TABLE" syntax to unify ACID ORC & Parquet
> support
> -
>
> Key: HIVE-18453
> URL: https://issues.apache.org/jira/browse/HIVE-18453
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Gopal V
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18453.01.patch, HIVE-18453.02.patch,
> HIVE-18453.03.patch, HIVE-18453.04.patch, HIVE-18453.05.patch,
> HIVE-18453.06.patch
>
>
> The ACID table markers are currently done with TBLPROPERTIES which is
> inherently fragile.
> The "create transactional table" offers a way to standardize the syntax and
> allows for future compatibility changes to support Parquet ACIDv2 tables
> along with ORC tables.
> The ACIDv2 design is format independent, with the ability to add new
> vectorized input formats with no changes to the design.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[
https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584880#comment-16584880
]
Hive QA commented on HIVE-20420:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 14 new + 1 unchanged - 0 fixed
= 15 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13328/dev-support/hive-personality.sh
|
| git revision | master / e57b52b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13328/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13328/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Provide a fallback authorizer when no other authorizer is in use
>
>
> Key: HIVE-20420
> URL: https://issues.apache.org/jira/browse/HIVE-20420
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20420.1.patch, HIVE-20420.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20422) TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
[
https://issues.apache.org/jira/browse/HIVE-20422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-20422:
Resolution: Fixed
Assignee: Ashutosh Chauhan
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Missed file rename
> TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
>
>
> Key: HIVE-20422
> URL: https://issues.apache.org/jira/browse/HIVE-20422
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Igor Kryvenko
>Assignee: Ashutosh Chauhan
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20422.patch
>
>
> Logs from Jenkins
> {code}
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
> type is not supported
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.getType(TypeConverter.java:135)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genTableLogicalPlan(CalcitePlanner.java:2920)
> ~[hive-ex$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4892)
> ~[hive-exec-4.$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1719)
> ~[hive-exec-4.0.0-SNAPS$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1667)
> ~[hive-exec-4.0.0-SNAPS$
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> ~[calcite-core-1.17.0.jar:1.17.0]
> {code}
> Command to reproduce it locally {{mvn clean test
> -Dtest=TestMiniLlapLocalCliDriver -Dqfile=udf_coalesce.q}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[ https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20420: -- Attachment: HIVE-20420.2.patch > Provide a fallback authorizer when no other authorizer is in use > > > Key: HIVE-20420 > URL: https://issues.apache.org/jira/browse/HIVE-20420 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20420.1.patch, HIVE-20420.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20422) TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
[
https://issues.apache.org/jira/browse/HIVE-20422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584866#comment-16584866
]
Hive QA commented on HIVE-20422:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936162/HIVE-20422.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13327/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13327/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13327/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-08-18 18:33:09.210
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-13327/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-08-18 18:33:09.214
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at e57b52b HIVE-20406 : Nested Coalesce giving incorrect results
(Vineet Garg via Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at e57b52b HIVE-20406 : Nested Coalesce giving incorrect results
(Vineet Garg via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-08-18 18:33:11.176
+ rm -rf ../yetus_PreCommit-HIVE-Build-13327
+ mkdir ../yetus_PreCommit-HIVE-Build-13327
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-13327
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-13327/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/ql/src/test/queries/clientpositive/udf_coalesce.q: does not exist in
index
error: patch failed: ql/src/test/queries/clientpositive/udf_coalesce.q:1
error: repository lacks the necessary blob to fall back on 3-way merge.
error: ql/src/test/queries/clientpositive/udf_coalesce.q: patch does not apply
error: src/test/queries/clientpositive/udf_coalesce.q: does not exist in index
error: src/test/results/clientpositive/udf_coalesce.q.out: does not exist in
index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-13327
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936162 - PreCommit-HIVE-Build
> TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
>
>
> Key: HIVE-20422
> URL: https://issues.apache.org/jira/browse/HIVE-20422
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20422.patch
>
>
> Logs from Jenkins
> {code}
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
> type is not supported
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
>
[jira] [Commented] (HIVE-20399) CTAS w/a custom table location that is not fully qualified fails for MM tables
[ https://issues.apache.org/jira/browse/HIVE-20399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584864#comment-16584864 ] Ashutosh Chauhan commented on HIVE-20399: - +1 pending tests > CTAS w/a custom table location that is not fully qualified fails for MM tables > -- > > Key: HIVE-20399 > URL: https://issues.apache.org/jira/browse/HIVE-20399 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20399.01.patch, HIVE-20399.02.patch, > HIVE-20399.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20422) TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
[
https://issues.apache.org/jira/browse/HIVE-20422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-20422:
Status: Patch Available (was: Open)
> TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
>
>
> Key: HIVE-20422
> URL: https://issues.apache.org/jira/browse/HIVE-20422
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20422.patch
>
>
> Logs from Jenkins
> {code}
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
> type is not supported
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.getType(TypeConverter.java:135)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genTableLogicalPlan(CalcitePlanner.java:2920)
> ~[hive-ex$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4892)
> ~[hive-exec-4.$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1719)
> ~[hive-exec-4.0.0-SNAPS$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1667)
> ~[hive-exec-4.0.0-SNAPS$
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> ~[calcite-core-1.17.0.jar:1.17.0]
> {code}
> Command to reproduce it locally {{mvn clean test
> -Dtest=TestMiniLlapLocalCliDriver -Dqfile=udf_coalesce.q}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20422) TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
[
https://issues.apache.org/jira/browse/HIVE-20422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-20422:
Attachment: HIVE-20422.patch
> TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
>
>
> Key: HIVE-20422
> URL: https://issues.apache.org/jira/browse/HIVE-20422
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20422.patch
>
>
> Logs from Jenkins
> {code}
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
> type is not supported
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.getType(TypeConverter.java:135)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genTableLogicalPlan(CalcitePlanner.java:2920)
> ~[hive-ex$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4892)
> ~[hive-exec-4.$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1719)
> ~[hive-exec-4.0.0-SNAPS$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1667)
> ~[hive-exec-4.0.0-SNAPS$
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> ~[calcite-core-1.17.0.jar:1.17.0]
> {code}
> Command to reproduce it locally {{mvn clean test
> -Dtest=TestMiniLlapLocalCliDriver -Dqfile=udf_coalesce.q}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16266) Enable function metadata to be written during bootstrap
[ https://issues.apache.org/jira/browse/HIVE-16266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584850#comment-16584850 ] Sushanth Sowmyan commented on HIVE-16266: - Hi [~akolb], apologies if this reply is no longer accurate ([~anishek] or [~sankarh] might be able to clarify if things have changed - I have not been active with hive for a year now), but at the time that the repl subsystem was written, that's correct, by intention. The basic idea is this - hive has two types of tables : MANAGED, where hive is responsible for the storage, and EXTERNAL, where some other external program is responsible for the storage. A key way to think about this distinction is what happens when you do a DROP TABLE. For MANAGED tables, if a DROP TABLE is issued, hive should delete the data on hdfs, since we own and manage the data as well. For EXTERNAL tables, we are guests, and some other tool is managing the data, and thus, we should not touch it - we can drop the metadata, but we leave the data on HDFS alone. Now, in the case where we're replicating from a primary to a secondary, if the table is a EXTERNAL table on the primary, then an external tool is managing it on the primary. But what about the secondary? Since the secondary is being "managed" by Hive Replication, and thus, Hive, we own and manage it, keeping it in sync with the primary. Thus, by definition, the copy is MANAGED even if the source is EXTERNAL. If we kept it EXTERNAL, we would start having some weird midway behaviour that we'd have to add complex rules for - consider the same deletion scenario: If we have a DROP PARTITION on the source table, by definition, on the source, we do not delete the data on source hdfs. The user will likely do a hdfs rm, refresh the data and might do a ADD PARTITION of new data. Now, what about the destination? Should we delete the data corresponding to that DROP PARTITION on destination? If so, then it is consistent with behaviour for MANAGED, rather than EXTERNAL, and thus, we should keep it as MANAGED. If not, then well, we have leftover data sitting in hdfs in the same location, and if new data gets added in, as a result of an upcoming ADD PARTITION, then the behaviour is indeterminable depending on the user - it can be the correct new data, it can be a partial merge or a weird append. That gets messy fast. So, for this problem and other possible unexpected problems, we decided to be consistent with the meaning of MANAGED and EXTERNAL, and always make repl destinations MANAGED. :) > Enable function metadata to be written during bootstrap > --- > > Key: HIVE-16266 > URL: https://issues.apache.org/jira/browse/HIVE-16266 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-16266.1.patch, HIVE-16266.2.patch, > HIVE-16266.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584846#comment-16584846
]
Igor Kryvenko edited comment on HIVE-20421 at 8/18/18 5:06 PM:
---
I've created follow up [JIRA|https://issues.apache.org/jira/browse/HIVE-20422]
was (Author: ikryvenko):
I've create follow up [JIRA|https://issues.apache.org/jira/browse/HIVE-20422]
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20422) TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
[
https://issues.apache.org/jira/browse/HIVE-20422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584845#comment-16584845
]
Igor Kryvenko commented on HIVE-20422:
--
FYI: [~vgarg]
> TestMiniLlapLocalCliDriver[udf_coalesce.q] is broken
>
>
> Key: HIVE-20422
> URL: https://issues.apache.org/jira/browse/HIVE-20422
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Igor Kryvenko
>Priority: Major
>
> Logs from Jenkins
> {code}
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
> type is not supported
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.getType(TypeConverter.java:135)
> ~[hive-exec-4.0.0-SNAP$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genTableLogicalPlan(CalcitePlanner.java:2920)
> ~[hive-ex$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4892)
> ~[hive-exec-4.$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1719)
> ~[hive-exec-4.0.0-SNAPS$
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1667)
> ~[hive-exec-4.0.0-SNAPS$
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> ~[calcite-core-1.17.0.jar:1.17.0]
> {code}
> Command to reproduce it locally {{mvn clean test
> -Dtest=TestMiniLlapLocalCliDriver -Dqfile=udf_coalesce.q}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584846#comment-16584846
]
Igor Kryvenko commented on HIVE-20421:
--
I've create follow up [JIRA|https://issues.apache.org/jira/browse/HIVE-20422]
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584844#comment-16584844
]
Igor Kryvenko edited comment on HIVE-20421 at 8/18/18 5:03 PM:
---
Actually, the failed test is not related. But it is not flaky.
Logs from Jenkins
{code}
org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
type is not supported
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.getType(TypeConverter.java:135)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genTableLogicalPlan(CalcitePlanner.java:2920)
~[hive-ex$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4892)
~[hive-exec-4.$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1719)
~[hive-exec-4.0.0-SNAPS$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1667)
~[hive-exec-4.0.0-SNAPS$
at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
~[calcite-core-1.17.0.jar:1.17.0]
{code}
Locally, {{mvn clean test -Dtest=TestMiniLlapLocalCliDriver
-Dqfile=udf_coalesce.q}} produces the same exception.
was (Author: ikryvenko):
Actually, the failed test is not related. But it is not flaky.
Logs from Jenkins
{code}
org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
type is not supported
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.getType(TypeConverter.java:135)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genTableLogicalPlan(CalcitePlanner.java:2920)
~[hive-ex$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4892)
~[hive-exec-4.$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1719)
~[hive-exec-4.0.0-SNAPS$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1667)
~[hive-exec-4.0.0-SNAPS$
at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
~[calcite-core-1.17.0.jar:1.17.0]
{code}
Locally, {{ mvn clean test -Dtest=TestMiniLlapLocalCliDriver
-Dqfile=udf_coalesce.q}} produces the same exception.
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
>
[jira] [Commented] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584844#comment-16584844
]
Igor Kryvenko commented on HIVE-20421:
--
Actually, the failed test is not related. But it is not flaky.
Logs from Jenkins
{code}
org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Union
type is not supported
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:271)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:160)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:255)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.convert(TypeConverter.java:154)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter.getType(TypeConverter.java:135)
~[hive-exec-4.0.0-SNAP$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genTableLogicalPlan(CalcitePlanner.java:2920)
~[hive-ex$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4892)
~[hive-exec-4.$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1719)
~[hive-exec-4.0.0-SNAPS$
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1667)
~[hive-exec-4.0.0-SNAPS$
at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
~[calcite-core-1.17.0.jar:1.17.0]
{code}
Locally, {{ mvn clean test -Dtest=TestMiniLlapLocalCliDriver
-Dqfile=udf_coalesce.q}} produces the same exception.
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584829#comment-16584829
]
Hive QA commented on HIVE-20421:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936159/HIVE-20421.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14885 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_coalesce]
(batchId=179)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13326/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13326/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13326/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936159 - PreCommit-HIVE-Build
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584812#comment-16584812
]
Hive QA commented on HIVE-20421:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13326/dev-support/hive-personality.sh
|
| git revision | master / e57b52b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13326/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at
[jira] [Commented] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584790#comment-16584790
]
Igor Kryvenko commented on HIVE-20421:
--
[~ashutoshc] [~vgarg] [~kgyrtkirk] Hi guys, could you review this small patch,
please?
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko reassigned HIVE-20421:
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-20421:
-
Status: Patch Available (was: Open)
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20421) Illegal character entity '\b' in hive-default.xml.template
[
https://issues.apache.org/jira/browse/HIVE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-20421:
-
Attachment: HIVE-20421.01.patch
> Illegal character entity '\b' in hive-default.xml.template
> --
>
> Key: HIVE-20421
> URL: https://issues.apache.org/jira/browse/HIVE-20421
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-20421.01.patch
>
>
> Illegal {{\b}} element in {{hive-site.xml. }}
> {code}
> TXN_OVERWRITE_X_LOCK("hive.txn.xlock.iow", true,
> "Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire
> Exclusive locks for\b" +
> "transactional tables. This ensures that inserts (w/o overwrite)
> running concurrently\n" +
> "are not hidden by the INSERT OVERWRITE."),
> {code}
> Seems like just typo of {{\n}}.
> Stacktrace
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion
> character (code 0x8
> at [row,col,system-id]:
> [3278,96,"file:/usr/local/apache-hive/conf/hive-site.xml"]
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5059)
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5132)
> at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5219)
> at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:5167)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[
https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584738#comment-16584738
]
Hive QA commented on HIVE-20420:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936129/HIVE-20420.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13325/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13325/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13325/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12936129/HIVE-20420.1.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936129 - PreCommit-HIVE-Build
> Provide a fallback authorizer when no other authorizer is in use
>
>
> Key: HIVE-20420
> URL: https://issues.apache.org/jira/browse/HIVE-20420
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20420.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[
https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584736#comment-16584736
]
Hive QA commented on HIVE-20420:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936129/HIVE-20420.1.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14893 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_coalesce]
(batchId=179)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13324/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13324/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13324/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936129 - PreCommit-HIVE-Build
> Provide a fallback authorizer when no other authorizer is in use
>
>
> Key: HIVE-20420
> URL: https://issues.apache.org/jira/browse/HIVE-20420
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20420.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[
https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584724#comment-16584724
]
Hive QA commented on HIVE-20420:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 14 new + 1 unchanged - 0 fixed
= 15 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13324/dev-support/hive-personality.sh
|
| git revision | master / e57b52b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13324/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13324/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Provide a fallback authorizer when no other authorizer is in use
>
>
> Key: HIVE-20420
> URL: https://issues.apache.org/jira/browse/HIVE-20420
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20420.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20377) Hive Kafka Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584717#comment-16584717
]
Hive QA commented on HIVE-20377:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936123/HIVE-20377.11.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13323/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13323/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13323/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12936123/HIVE-20377.11.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936123 - PreCommit-HIVE-Build
> Hive Kafka Storage Handler
> --
>
> Key: HIVE-20377
> URL: https://issues.apache.org/jira/browse/HIVE-20377
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20377.10.patch, HIVE-20377.11.patch,
> HIVE-20377.4.patch, HIVE-20377.5.patch, HIVE-20377.6.patch,
> HIVE-20377.8.patch, HIVE-20377.8.patch, HIVE-20377.patch
>
>
> h1. Goal
> * Read streaming data form Kafka queue as an external table.
> * Allow streaming navigation by pushing down filters on Kafka record
> partition id, offset and timestamp.
> * Insert streaming data form Kafka to an actual Hive internal table, using
> CTAS statement.
> h1. Example
> h2. Create the external table
> {code}
> CREATE EXTERNAL TABLE kafka_table (`timestamp` timestamp, page string, `user`
> string, language string, added int, deleted int, flags string,comment string,
> namespace string)
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES
> ("kafka.topic" = "wikipedia",
> "kafka.bootstrap.servers"="brokeraddress:9092",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.JsonSerDe");
> {code}
> h2. Kafka Metadata
> In order to keep track of Kafka records the storage handler will add
> automatically the Kafka row metadata eg partition id, record offset and
> record timestamp.
> {code}
> DESCRIBE EXTENDED kafka_table
> timestamp timestamp from deserializer
> page string from deserializer
> user string from deserializer
> language string from deserializer
> country string from deserializer
> continent string from deserializer
> namespace string from deserializer
> newpage boolean from deserializer
> unpatrolled boolean from deserializer
> anonymous boolean from deserializer
> robot boolean from deserializer
> added int from deserializer
> deleted int from deserializer
> delta bigint from deserializer
> __partition int from deserializer
> __offset bigint from deserializer
> __timestamp bigint from deserializer
> {code}
> h2. Filter push down.
> Newer Kafka consumers 0.11.0 and higher allow seeking on the stream based on
> a given offset. The proposed storage handler will be able to leverage such
> API by pushing down filters over metadata columns, namely __partition (int),
> __offset(long) and __timestamp(long)
> For instance Query like
> {code}
> select `__offset` from kafka_table where (`__offset` < 10 and `__offset`>3
> and `__partition` = 0) or (`__partition` = 0 and `__offset` < 105 and
> `__offset` > 99) or (`__offset` = 109);
> {code}
> Will result on a scan of partition 0 only then read only records between
> offset 4 and 109.
> h2. With timestamp seeks
> The seeking based on the internal timestamps allows the handler to run on
> recently arrived data, by doing
> {code}
> select count(*) from kafka_table where `__timestamp` > 1000 *
> to_unix_timestamp(CURRENT_TIMESTAMP - interval '20' hours) ;
> {code}
> This allows for implicit relationships between event timestamps and kafka
> timestamps to be expressed in queries (i.e event_timestamp is always < than
> kafka __timestamp and kafka __timestamp is never > 15 minutes from event etc).
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HIVE-20370) Vectorization: Add Native Vector MapJoin hash table optimization for Left/Right Outer Joins when there are no Small Table values
[
https://issues.apache.org/jira/browse/HIVE-20370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584716#comment-16584716
]
Hive QA commented on HIVE-20370:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936121/HIVE-20370.04.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14885 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_coalesce]
(batchId=179)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13322/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13322/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13322/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936121 - PreCommit-HIVE-Build
> Vectorization: Add Native Vector MapJoin hash table optimization for
> Left/Right Outer Joins when there are no Small Table values
>
>
> Key: HIVE-20370
> URL: https://issues.apache.org/jira/browse/HIVE-20370
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-20370.01.patch, HIVE-20370.02.patch,
> HIVE-20370.03.patch, HIVE-20370.04.patch
>
>
> Similar to Native Vector MapJoin's InnerBigOnly optimization that uses an
> efficient Hash Multi-Set with a counter instead of a Hash Map with an empty
> value, do the same for Outer joins.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20370) Vectorization: Add Native Vector MapJoin hash table optimization for Left/Right Outer Joins when there are no Small Table values
[
https://issues.apache.org/jira/browse/HIVE-20370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584707#comment-16584707
]
Hive QA commented on HIVE-20370:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
31s{color} | {color:red} branch/itests/hive-jmh cannot run convertXmlToText
from findbugs {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
21s{color} | {color:red} hive-jmh in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} itests/hive-jmh: The patch generated 21 new + 90
unchanged - 0 fixed = 111 total (was 90) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
51s{color} | {color:red} ql: The patch generated 105 new + 973 unchanged - 57
fixed = 1078 total (was 1030) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
14s{color} | {color:red} patch/itests/hive-jmh cannot run convertXmlToText from
findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13322/dev-support/hive-personality.sh
|
| git revision | master / e57b52b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13322/yetus/branch-findbugs-itests_hive-jmh.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13322/yetus/patch-mvninstall-itests_hive-jmh.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13322/yetus/diff-checkstyle-itests_hive-jmh.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13322/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13322/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13322/yetus/patch-findbugs-itests_hive-jmh.txt
|
| modules | C: itests/hive-jmh ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13322/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Add Native Vector MapJoin hash table optimization for
> Left/Right Outer Joins when there are no Small Table values
>
[jira] [Commented] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584695#comment-16584695
]
Hive QA commented on HIVE-17979:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12896290/HIVE-17979.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14880 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
[druidmini_dynamic_partition.q,druidmini_test_ts.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test_insert.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_coalesce]
(batchId=179)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13321/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13321/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13321/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12896290 - PreCommit-HIVE-Build
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20419) Vectorization: Prevent mutation of VectorPartitionDesc after being used in a hashmap key
[
https://issues.apache.org/jira/browse/HIVE-20419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-20419:
---
Description:
This is going into the loop because the VectorPartitionDesc is modified after
it is used in the HashMap key - resulting in a hashcode & equals modification
after it has been placed in the hashmap.
{code}
HiveServer2-Background-Pool: Thread-6049 State: RUNNABLE CPU usage on sample:
621ms
java.util.HashMap$TreeNode.find(int, Object, Class) HashMap.java:1869 <7
recursive calls>
java.util.HashMap$TreeNode.putTreeVal(HashMap, HashMap$Node[], int, Object,
Object) HashMap.java:1989
java.util.HashMap.putVal(int, Object, Object, boolean, boolean) HashMap.java:637
java.util.HashMap.put(Object, Object) HashMap.java:611
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.addVectorPartitionDesc(PartitionDesc,
VectorPartitionDesc, Map) Vectorizer.java:1272
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(PartitionDesc,
boolean, List, Set, Map, Set, ArrayList, Set) Vectorizer.java:1323
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(MapWork,
String, TableScanOperator, Vectorizer$VectorTaskColumnInfo)
Vectorizer.java:1654
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(MapWork,
Vectorizer$VectorTaskColumnInfo, boolean) Vectorizer.java:1865
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(MapWork,
boolean) Vectorizer.java:1109
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Node,
Stack, Object[]) Vectorizer.java:961
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(Node, Stack,
TaskGraphWalker$TaskGraphWalkerContext) TaskGraphWalker.java:111
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(Node)
TaskGraphWalker.java:180
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(Collection, HashMap)
TaskGraphWalker.java:125
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(PhysicalContext)
Vectorizer.java:2442
org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(List,
ParseContext, Context) TezCompiler.java:717
org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(ParseContext, List,
HashSet, HashSet) TaskCompiler.java:258
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(ASTNode,
SemanticAnalyzer$PlannerContextFactory) SemanticAnalyzer.java:12443
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(ASTNode)
CalcitePlanner.java:358
{code}
was:
With ACID table, the format and schema layouts are much more strictly
controlled - the table cannot be made of partial ORC and partial RCFile.
This assumption can remove this loop and the slow check for schema between each
partition before vectorizing the operators - the worst-case performance is the
common & correct case, where all of them match.
{code}
HiveServer2-Background-Pool: Thread-6049 State: RUNNABLE CPU usage on sample:
621ms
java.util.HashMap$TreeNode.find(int, Object, Class) HashMap.java:1869 <7
recursive calls>
java.util.HashMap$TreeNode.putTreeVal(HashMap, HashMap$Node[], int, Object,
Object) HashMap.java:1989
java.util.HashMap.putVal(int, Object, Object, boolean, boolean) HashMap.java:637
java.util.HashMap.put(Object, Object) HashMap.java:611
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.addVectorPartitionDesc(PartitionDesc,
VectorPartitionDesc, Map) Vectorizer.java:1272
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(PartitionDesc,
boolean, List, Set, Map, Set, ArrayList, Set) Vectorizer.java:1323
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(MapWork,
String, TableScanOperator, Vectorizer$VectorTaskColumnInfo)
Vectorizer.java:1654
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(MapWork,
Vectorizer$VectorTaskColumnInfo, boolean) Vectorizer.java:1865
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(MapWork,
boolean) Vectorizer.java:1109
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Node,
Stack, Object[]) Vectorizer.java:961
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(Node, Stack,
TaskGraphWalker$TaskGraphWalkerContext) TaskGraphWalker.java:111
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(Node)
TaskGraphWalker.java:180
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(Collection, HashMap)
TaskGraphWalker.java:125
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(PhysicalContext)
Vectorizer.java:2442
[jira] [Updated] (HIVE-20419) Vectorization: Prevent mutation of VectorPartitionDesc after being used in a hashmap key
[
https://issues.apache.org/jira/browse/HIVE-20419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-20419:
---
Summary: Vectorization: Prevent mutation of VectorPartitionDesc after being
used in a hashmap key (was: Vectorization: Speed up
VectorizationDispatcher.validateInputFormatAndSchemaEvolution() for ACIDv2)
> Vectorization: Prevent mutation of VectorPartitionDesc after being used in a
> hashmap key
>
>
> Key: HIVE-20419
> URL: https://issues.apache.org/jira/browse/HIVE-20419
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Gopal V
>Priority: Major
>
> With ACID table, the format and schema layouts are much more strictly
> controlled - the table cannot be made of partial ORC and partial RCFile.
> This assumption can remove this loop and the slow check for schema between
> each partition before vectorizing the operators - the worst-case performance
> is the common & correct case, where all of them match.
> {code}
> HiveServer2-Background-Pool: Thread-6049 State: RUNNABLE CPU usage on sample:
> 621ms
> java.util.HashMap$TreeNode.find(int, Object, Class) HashMap.java:1869 <7
> recursive calls>
> java.util.HashMap$TreeNode.putTreeVal(HashMap, HashMap$Node[], int, Object,
> Object) HashMap.java:1989
> java.util.HashMap.putVal(int, Object, Object, boolean, boolean)
> HashMap.java:637
> java.util.HashMap.put(Object, Object) HashMap.java:611
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.addVectorPartitionDesc(PartitionDesc,
> VectorPartitionDesc, Map) Vectorizer.java:1272
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(PartitionDesc,
> boolean, List, Set, Map, Set, ArrayList, Set) Vectorizer.java:1323
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(MapWork,
> String, TableScanOperator, Vectorizer$VectorTaskColumnInfo)
> Vectorizer.java:1654
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(MapWork,
> Vectorizer$VectorTaskColumnInfo, boolean) Vectorizer.java:1865
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(MapWork,
> boolean) Vectorizer.java:1109
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Node,
> Stack, Object[]) Vectorizer.java:961
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(Node, Stack,
> TaskGraphWalker$TaskGraphWalkerContext) TaskGraphWalker.java:111
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(Node)
> TaskGraphWalker.java:180
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(Collection,
> HashMap) TaskGraphWalker.java:125
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(PhysicalContext)
> Vectorizer.java:2442
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(List,
> ParseContext, Context) TezCompiler.java:717
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(ParseContext, List,
> HashSet, HashSet) TaskCompiler.java:258
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(ASTNode,
> SemanticAnalyzer$PlannerContextFactory) SemanticAnalyzer.java:12443
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(ASTNode)
> CalcitePlanner.java:358
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying
[
https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584687#comment-16584687
]
Hive QA commented on HIVE-17979:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 136 unchanged - 0
fixed = 137 total (was 136) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13321/dev-support/hive-personality.sh
|
| git revision | master / e57b52b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13321/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13321/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13321/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez: Improve ReduceRecordSource passDownKey copying
> ---
>
> Key: HIVE-17979
> URL: https://issues.apache.org/jira/browse/HIVE-17979
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each
> input gets its own ReduceRecordSource
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the
> tag is not part of the Key byte stream), this means for a 2-table join there
> are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key,
> List has more than 1 value in it. Otherwise the copy is entirely
> wasted CPU cycles, because it deserializes the entire row to extract the key
> and discards the row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20418) LLAP IO may not handle ORC files with row index disabled correctly for queries with no columns selected
[
https://issues.apache.org/jira/browse/HIVE-20418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584678#comment-16584678
]
Hive QA commented on HIVE-20418:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936111/HIVE-20418.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13320/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13320/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13320/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12936111/HIVE-20418.01.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936111 - PreCommit-HIVE-Build
> LLAP IO may not handle ORC files with row index disabled correctly for
> queries with no columns selected
> ---
>
> Key: HIVE-20418
> URL: https://issues.apache.org/jira/browse/HIVE-20418
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20418.01.patch, HIVE-20418.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20418) LLAP IO may not handle ORC files with row index disabled correctly for queries with no columns selected
[
https://issues.apache.org/jira/browse/HIVE-20418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584677#comment-16584677
]
Hive QA commented on HIVE-20418:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12936111/HIVE-20418.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14886 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_coalesce]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_acid4]
(batchId=158)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13319/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13319/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13319/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12936111 - PreCommit-HIVE-Build
> LLAP IO may not handle ORC files with row index disabled correctly for
> queries with no columns selected
> ---
>
> Key: HIVE-20418
> URL: https://issues.apache.org/jira/browse/HIVE-20418
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20418.01.patch, HIVE-20418.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20418) LLAP IO may not handle ORC files with row index disabled correctly for queries with no columns selected
[
https://issues.apache.org/jira/browse/HIVE-20418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584659#comment-16584659
]
Hive QA commented on HIVE-20418:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 2307 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 2 new + 115 unchanged - 1
fixed = 117 total (was 116) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13319/dev-support/hive-personality.sh
|
| git revision | master / e57b52b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13319/yetus/diff-checkstyle-ql.txt
|
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13319/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP IO may not handle ORC files with row index disabled correctly for
> queries with no columns selected
> ---
>
> Key: HIVE-20418
> URL: https://issues.apache.org/jira/browse/HIVE-20418
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20418.01.patch, HIVE-20418.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
