[jira] [Updated] (HIVE-19564) Vectorization: Fix NULL / Wrong Results issues in Arithmetic
[ https://issues.apache.org/jira/browse/HIVE-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19564: Fix Version/s: 4.0.0 > Vectorization: Fix NULL / Wrong Results issues in Arithmetic > > > Key: HIVE-19564 > URL: https://issues.apache.org/jira/browse/HIVE-19564 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 4.0.0 > > Attachments: HIVE-19564.01.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized arithmetic (+,-,*,/,%) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19564) Vectorization: Fix NULL / Wrong Results issues in Arithmetic
[ https://issues.apache.org/jira/browse/HIVE-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19564: Target Version/s: (was: 3.1.0) > Vectorization: Fix NULL / Wrong Results issues in Arithmetic > > > Key: HIVE-19564 > URL: https://issues.apache.org/jira/browse/HIVE-19564 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19564.01.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized arithmetic (+,-,*,/,%) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HIVE-19734) Beeline: When beeline-site.xml is present, beeline does not honor -n (username) and -p (password) arguments
[ https://issues.apache.org/jira/browse/HIVE-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair reopened HIVE-19734: -- > Beeline: When beeline-site.xml is present, beeline does not honor -n > (username) and -p (password) arguments > --- > > Key: HIVE-19734 > URL: https://issues.apache.org/jira/browse/HIVE-19734 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Thejas M Nair >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19734.1.patch, HIVE-19734.2-branch-3.patch, > HIVE-19734.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-19921: - Attachment: HIVE-19921.02-branch-3.patch > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01-branch-3.patch, HIVE-19921.01.patch, > HIVE-19921.02-branch-3.patch > > > The perf log should return duration instead of end time. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19267) Create/Replicate ACID Write event
[ https://issues.apache.org/jira/browse/HIVE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-19267: --- Attachment: (was: HIVE-19267.17.patch) > Create/Replicate ACID Write event > - > > Key: HIVE-19267 > URL: https://issues.apache.org/jira/browse/HIVE-19267 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Attachments: HIVE-19267.01.patch, HIVE-19267.02.patch, > HIVE-19267.03.patch, HIVE-19267.04.patch, HIVE-19267.05.patch, > HIVE-19267.06.patch, HIVE-19267.07.patch, HIVE-19267.08.patch, > HIVE-19267.09.patch, HIVE-19267.10.patch, HIVE-19267.11.patch, > HIVE-19267.12.patch, HIVE-19267.13.patch, HIVE-19267.14.patch, > HIVE-19267.15.patch, HIVE-19267.16.patch, HIVE-19267.17.patch > > > > h1. Replicate ACID write Events > * Create new EVENT_WRITE event with related message format to log the write > operations with in a txn along with data associated. > * Log this event when perform any writes (insert into, insert overwrite, > load table, delete, update, merge, truncate) on table/partition. > * If a single MERGE/UPDATE/INSERT/DELETE statement operates on multiple > partitions, then need to log one event per partition. > * DbNotificationListener should log this type of event to special metastore > table named "MTxnWriteNotificationLog". > * This table should maintain a map of txn ID against list of > tables/partitions written by given txn. > * The entry for a given txn should be removed by the cleaner thread that > removes the expired events from EventNotificationTable. > h1. Replicate Commit Txn operation (with writes) > Add new EVENT_COMMIT_TXN to log the metadata/data of all tables/partitions > modified within the txn. > *Source warehouse:* > * This event should read the EVENT_WRITEs from "MTxnWriteNotificationLog" > metastore table to consolidate the list of tables/partitions modified within > this txn scope. > * Based on the list of tables/partitions modified and table Write ID, need > to compute the list of delta files added by this txn. > * Repl dump should read this message and dump the metadata and delta files > list. > *Target warehouse:* > * Ensure snapshot isolation at target for on-going read txns which shouldn't > view the data replicated from committed txn. (Ensured with open and allocate > write ID events). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19812) Disable external table replication by default via a configuration property
[
https://issues.apache.org/jira/browse/HIVE-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518980#comment-16518980
]
Hive QA commented on HIVE-19812:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 110
unchanged - 0 fixed = 112 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11961/dev-support/hive-personality.sh
|
| git revision | master / e6577a0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11961/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: common itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11961/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Disable external table replication by default via a configuration property
> --
>
> Key: HIVE-19812
> URL: https://issues.apache.org/jira/browse/HIVE-19812
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19812.01.patch, HIVE-19812.02.patch,
> HIVE-19812.03.patch, HIVE-19812.04.patch, HIVE-19812.05.patch,
> HIVE-19812.06.patch
>
>
> use a hive config property to allow external table replication. set this
> property by default to prevent external table replication.
>
[jira] [Updated] (HIVE-19890) ACID: Inherit bucket-id from original ROW_ID for delete deltas
[ https://issues.apache.org/jira/browse/HIVE-19890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-19890: --- Attachment: HIVE-19890.3.patch > ACID: Inherit bucket-id from original ROW_ID for delete deltas > -- > > Key: HIVE-19890 > URL: https://issues.apache.org/jira/browse/HIVE-19890 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-19890.1.patch, HIVE-19890.2.patch, > HIVE-19890.3.patch > > > The ACID delete deltas for unbucketed tables are written to arbitrary files, > which should instead be shuffled using the bucket-id instead of hash(ROW__ID). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-12192) Hive should carry out timestamp computations in UTC
[
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-12192:
---
Attachment: HIVE-12192.17.patch
> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Labels: timestamp
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch,
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch,
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch,
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch,
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch,
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch,
> HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles
> that alternates between PST and PDT, there are times that cannot be
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a
> java.sql.Timestamp avoids this bug, while still returning correct values for
> {{getYear}} etc. Using UTC as the convenience representation (timestamp
> without time zone has no real zone) would make timestamp calculations more
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the
> result is with respect to ["the default timezone and default
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
> That function would need to be updated to use the
> {{System.getProperty("user.timezone")}} zone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19911) Hive delete queries fail with Invalid table alias or column reference
[
https://issues.apache.org/jira/browse/HIVE-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518942#comment-16518942
]
Hive QA commented on HIVE-19911:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928374/HIVE-19911.1-branch-2.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 10579 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin2] (batchId=51)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[constprog_semijoin]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[constprog_semijoin]
(batchId=165)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join12]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[mapreduce2]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[semijoin]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoin]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt9]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_3]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_4]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats10]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union5]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_10]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_left_outer_join]
(batchId=107)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=263)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testNonAcidToAcidConversion02
(batchId=275)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion02
(batchId=272)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11960/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11960/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11960/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928374 - PreCommit-HIVE-Build
> Hive delete queries fail with Invalid table alias or column reference
> -
>
> Key: HIVE-19911
> URL: https://issues.apache.org/jira/browse/HIVE-19911
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.3.3
>Reporter: Mykhailo Kysliuk
>Assignee: Mykhailo Kysliuk
>Priority: Major
> Attachments: HIVE-19911.1-branch-2.3.patch
>
>
> Env:
> hadoop-2.7.0
> hive-2.3.3
> OS:
> centos-release-7-5.1804.el7.centos.x86_64
> Steps to reproduce (at hive cli):
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> DROP TABLE IF EXISTS detaillineitem_all;
> DROP TABLE IF EXISTS detaillineitem_all_delete_1526330755128;
> CREATE TABLE `detaillineitem_all`(
> `detailid` decimal(20,0),
> `branchnumber` varchar(3)
> ) PARTITIONED BY (
> `branchnumber_p` varchar(3))
> CLUSTERED BY (
> detailid)
> INTO 25 BUCKETS
> STORED AS ORC
> TBLPROPERTIES (
> 'orc.compress'='NONE',
> 'transactional'='true');
> CREATE TABLE `detaillineitem_all_delete_1526330755128`(
> `detailid` decimal(20,0),
> `branchnumber` varchar(3),
> `branchnumber_p` varchar(3));
> DELETE from detaillineitem_all WHERE EXISTS (
> SELECT
> 1
> FROM
> detaillineitem_all_delete_1526330755128 AS t1
> WHERE
> (detaillineitem_all.detailid = t1.detailid)
> AND
> (detaillineitem_all.branchnumber = CAST(t1.branchnumber AS STRING)));
> {code}
> Exception:
> {code}
> 2018-06-15T16:51:48,625 ERROR [f6bd86a7-04e5-4284-9031-3b9a0ccc80f3 main]
> ql.Driver: FAILED: SemanticException Line 0:-1 Invalid table alias or column
> reference 'sq_1': (possible column names are: mber)) sq_corr_1)) (tok_where
> (= 1 1), (. (tok_table_or_col sq_1) sq_corr_1))
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 0:-1 Invalid table
> alias or column reference 'sq_1': (possible column names
[jira] [Updated] (HIVE-19882) Fix QTestUtil session lifecycle
[
https://issues.apache.org/jira/browse/HIVE-19882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-19882:
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
pushed to master. Thank you Jason for reviewing the changes.
This should stabilize (at least) 2 falkiness sources
> Fix QTestUtil session lifecycle
> ---
>
> Key: HIVE-19882
> URL: https://issues.apache.org/jira/browse/HIVE-19882
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19882.01.patch, HIVE-19882.02.patch,
> HIVE-19882.03.patch, HIVE-19882.04.patch, HIVE-19882.05.patch,
> HIVE-19882.06.patch, HIVE-19882.07.patch
>
>
> there are a number of strange come and go failing tests; it was always
> strange to me that qtestutil cleans up at some questionable points - this
> seems to be leading to executing some commands with the previous qfiles
> session...
> ideally the session/etc should start/reused in {{before}}
> and it should be closed in {{after}}
> seems like configuration is handled probably incorrectly; saving the conf
> after initialization - and restoring it for a new session should ensure
> consistency
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19928) Load Data for managed tables should set the owner of loaded files to a configurable user
[ https://issues.apache.org/jira/browse/HIVE-19928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-19928: -- Attachment: HIVE-19928.5.patch > Load Data for managed tables should set the owner of loaded files to a > configurable user > > > Key: HIVE-19928 > URL: https://issues.apache.org/jira/browse/HIVE-19928 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19928.1.patch, HIVE-19928.2.patch, > HIVE-19928.3.patch, HIVE-19928.4.patch, HIVE-19928.5.patch > > > load data of managed tables should set the owner of the loaded files to a > configurable user. the default user should be hive. > If the owner of existing file is not hive, then a rename/move operation > should be replaced by copy with the copied file having hive as owner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-18725:
-
Attachment: HIVE-18725.08.patch
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch,
> HIVE-18725.06.patch, HIVE-18725.07.patch, HIVE-18725.08.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518920#comment-16518920
]
Igor Kryvenko commented on HIVE-18725:
--
Seems *{{TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]}}*
fail is not related. Locally it works fine. If it is flaky I will create an
issue to disable it.
I will reattach the same patch again.
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch,
> HIVE-18725.06.patch, HIVE-18725.07.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19267) Create/Replicate ACID Write event
[
https://issues.apache.org/jira/browse/HIVE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518912#comment-16518912
]
Hive QA commented on HIVE-19267:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928367/HIVE-19267.17.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 14524 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestClearDanglingScratchDir - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestStorageBasedMetastoreAuthorizationProvider - did not produce a TEST-*.xml
file (likely timed out) (batchId=236)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11959/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11959/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11959/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928367 - PreCommit-HIVE-Build
> Create/Replicate ACID Write event
> -
>
> Key: HIVE-19267
> URL: https://issues.apache.org/jira/browse/HIVE-19267
> Project: Hive
> Issue Type: Sub-task
> Components: repl, Transactions
>Affects Versions: 3.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: ACID, DR, pull-request-available, replication
> Attachments: HIVE-19267.01.patch, HIVE-19267.02.patch,
> HIVE-19267.03.patch, HIVE-19267.04.patch, HIVE-19267.05.patch,
> HIVE-19267.06.patch, HIVE-19267.07.patch, HIVE-19267.08.patch,
> HIVE-19267.09.patch, HIVE-19267.10.patch, HIVE-19267.11.patch,
> HIVE-19267.12.patch, HIVE-19267.13.patch, HIVE-19267.14.patch,
> HIVE-19267.15.patch, HIVE-19267.16.patch, HIVE-19267.17.patch,
> HIVE-19267.17.patch
>
>
>
> h1. Replicate ACID write Events
> * Create new EVENT_WRITE event with related message format to log the write
> operations with in a txn along with data associated.
> * Log this event when perform any writes (insert into, insert overwrite,
> load table, delete, update, merge, truncate) on table/partition.
> * If a single MERGE/UPDATE/INSERT/DELETE statement operates on multiple
> partitions, then need to log one event per partition.
> * DbNotificationListener should log this type of event to special metastore
> table named "MTxnWriteNotificationLog".
> * This table should maintain a map of txn ID against list of
> tables/partitions written by given txn.
> * The entry for a given txn should be removed by the cleaner thread that
> removes the expired events from EventNotificationTable.
> h1. Replicate Commit Txn operation (with writes)
> Add new EVENT_COMMIT_TXN to log the metadata/data of all tables/partitions
> modified within the txn.
> *Source warehouse:*
> * This event should read the EVENT_WRITEs from "MTxnWriteNotificationLog"
> metastore table to consolidate the list of tables/partitions modified within
> this txn scope.
> * Based on the list of tables/partitions modified and table Write ID, need
> to compute the list of delta files added by this txn.
> * Repl dump should read this message and dump the metadata and delta files
> list.
> *Target warehouse:*
> * Ensure snapshot isolation at target for on-going read txns which shouldn't
> view the data replicated from committed txn. (Ensured with open and allocate
> write ID events).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19850) Dynamic partition pruning in Tez is leading to 'No work found for tablescan' error
[
https://issues.apache.org/jira/browse/HIVE-19850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-19850:
--
Description:
When multiple views are used along with union all, it is resulting in the
following error when dynamic partition pruning is enabled in tez.
{code:java}
Exception in thread "main" java.lang.AssertionError: No work found for
tablescan TS[8]
at
org.apache.hadoop.hive.ql.parse.GenTezUtils.processAppMasterEvent(GenTezUtils.java:408)
at
org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:383)
at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:205)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10371)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:208)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:239)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:347)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1203)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1257)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1140)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1130)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:204)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:433)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:894)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:825)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:726)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:223)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136){code}
*Steps to reproduce:*
set hive.execution.engine=tez;
set hive.tez.dynamic.partition.pruning=true;
CREATE TABLE t1(key string, value string, c_int int, c_float float, c_boolean
boolean) partitioned by (dt string);
CREATE TABLE t2(key string, value string, c_int int, c_float float, c_boolean
boolean) partitioned by (dt string);
CREATE TABLE t3(key string, value string, c_int int, c_float float, c_boolean
boolean) partitioned by (dt string);
insert into table t1 partition(dt='2018') values ('k1','v1',1,1.0,true);
insert into table t2 partition(dt='2018') values ('k2','v2',2,2.0,true);
insert into table t3 partition(dt='2018') values ('k3','v3',3,3.0,true);
CREATE VIEW `view1` AS select
`t1`.`key`,`t1`.`value`,`t1`.`c_int`,`t1`.`c_float`,`t1`.`c_boolean`,`t1`.`dt`
from `t1` union all select
`t2`.`key`,`t2`.`value`,`t2`.`c_int`,`t2`.`c_float`,`t2`.`c_boolean`,`t2`.`dt`
from `t2`;
CREATE VIEW `view2` AS select
`t2`.`key`,`t2`.`value`,`t2`.`c_int`,`t2`.`c_float`,`t2`.`c_boolean`,`t2`.`dt`
from `t2` union all select
`t3`.`key`,`t3`.`value`,`t3`.`c_int`,`t3`.`c_float`,`t3`.`c_boolean`,`t3`.`dt`
from `t3`;
create table t4 as select key,value,c_int,c_float,c_boolean,dt from t1 union
all select v1.key,v1.value,v1.c_int,v1.c_float,v1.c_boolean,v1.dt from view1 v1
join view2 v2 on v1.dt=v2.dt;
CREATE VIEW `view3` AS select
`t4`.`key`,`t4`.`value`,`t4`.`c_int`,`t4`.`c_float`,`t4`.`c_boolean`,`t4`.`dt`
from `t4` union all select
`t1`.`key`,`t1`.`value`,`t1`.`c_int`,`t1`.`c_float`,`t1`.`c_boolean`,`t1`.`dt`
from `t1`;
select count(0) from view2 v2 join view3 v3 on v2.dt=v3.dt; // Throws No work
found for tablescan error
was:
When multiple views are used along with union all, it is resulting in the
following error when dynamic partition pruning is enabled in tez.
{code:java}
Exception in thread "main" java.lang.AssertionError: No work found for
tablescan TS[8]
at
org.apache.hadoop.hive.ql.parse.GenTezUtils.processAppMasterEvent(GenTezUtils.java:408)
at
org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:383)
at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:205)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10371)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:208)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:239)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:347)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1203)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1257)
at org.apache.had
[jira] [Commented] (HIVE-19267) Create/Replicate ACID Write event
[
https://issues.apache.org/jira/browse/HIVE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518903#comment-16518903
]
Hive QA commented on HIVE-19267:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/server-extensions in master has 4 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
12s{color} | {color:blue} standalone-metastore in master has 227 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
34s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} hcatalog/server-extensions: The patch generated 0
new + 4 unchanged - 3 fixed = 4 total (was 7) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} itests/hive-unit: The patch generated 34 new + 650
unchanged - 5 fixed = 684 total (was 655) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
51s{color} | {color:red} ql: The patch generated 4 new + 1301 unchanged - 31
fixed = 1305 total (was 1332) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} standalone-metastore: The patch generated 7 new + 2187
unchanged - 6 fixed = 2194 total (was 2193) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 43 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
33s{color} | {color:red} hcatalog/server-extensions generated 1 new + 2
unchanged - 2 fixed = 3 total (was 4) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
14s{color} | {color:red} ql generated 1 new + 2279 unchanged - 1 fixed = 2280
total (was 2280) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {
[jira] [Commented] (HIVE-19829) Incremental replication load should create tasks in execution phase rather than semantic phase
[
https://issues.apache.org/jira/browse/HIVE-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518878#comment-16518878
]
Hive QA commented on HIVE-19829:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928368/HIVE-19829.07.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 14462 tests
executed
*Failed tests:*
{noformat}
TestBeeLineExceptionHandling - did not produce a TEST-*.xml file (likely timed
out) (batchId=193)
TestBeeLineHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestBeelineArgParsing - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestClientCommandHookFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=193)
TestHiveCli - did not produce a TEST-*.xml file (likely timed out) (batchId=193)
TestHiveSchemaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestIncrementalRows - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestShutdownHook - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestTableOutputFormat - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11958/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11958/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11958/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928368 - PreCommit-HIVE-Build
> Incremental replication load should create tasks in execution phase rather
> than semantic phase
> --
>
> Key: HIVE-19829
> URL: https://issues.apache.org/jira/browse/HIVE-19829
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19829.01.patch, HIVE-19829.02.patch,
> HIVE-19829.03.patch, HIVE-19829.04.patch, HIVE-19829.06.patch,
> HIVE-19829.07.patch, HIVE-19829.07.patch
>
>
> Split the incremental load into multiple iterations. In each iteration create
> number of tasks equal to the configured value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19829) Incremental replication load should create tasks in execution phase rather than semantic phase
[
https://issues.apache.org/jira/browse/HIVE-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518859#comment-16518859
]
Hive QA commented on HIVE-19829:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
36s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 4 new + 122
unchanged - 4 fixed = 126 total (was 126) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 10 new + 178 unchanged - 8
fixed = 188 total (was 186) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
10s{color} | {color:red} ql generated 4 new + 2277 unchanged - 3 fixed = 2281
total (was 2280) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | org.apache.hadoop.hive.ql.exec.repl.ReplLoadWork is Serializable;
consider declaring a serialVersionUID At ReplLoadWork.java:a serialVersionUID
At ReplLoadWork.java:[lines 42-111] |
| |
org.apache.hadoop.hive.ql.exec.repl.incremental.IncrementalLoadTasksBuilder.numIteration
should be package protected In IncrementalLoadTasksBuilder.java: In
IncrementalLoadTasksBuilder.java |
| | Write to static field
org.apache.hadoop.hive.ql.exec.repl.incremental.IncrementalLoadTasksBuilder.numIteration
from instance method new
org.apache.hadoop.hive.ql.exec.repl.incremental.IncrementalLoadTasksBuilder(String,
String, String, IncrementalLoadEventsIterator, HiveConf) At
IncrementalLoadTasksBuilder.java:from instance method new
org.apache.hadoop.hive.ql.exec.repl.incremental.IncrementalLoadTasksBuilder(String,
String, String, IncrementalLoadEventsIterator, HiveConf) At
IncrementalLoadTasksBuilder.java:[line 83] |
| | Write to static field
org.apache.hadoop.hive.ql.exec.repl.incremental.IncrementalLoadTasksBuilder.numIteration
from instance method
org.apache.hadoop.hive.ql.exec.repl.incremental.IncrementalLoadTasksBuilder.build(DriverContext,
Hive, Logger) At IncrementalLoadTasksBuilder.java:from instance method
org.apache.hadoop.hive.ql.exec.repl.incremental.IncrementalLoadTasksBuilder.build(DriverContext,
Hive, Logger) At IncrementalLoadTasksBuilder.java:[line 92] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-
[jira] [Commented] (HIVE-12342) Set default value of hive.optimize.index.filter to true
[
https://issues.apache.org/jira/browse/HIVE-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518824#comment-16518824
]
Hive QA commented on HIVE-12342:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928354/HIVE-12342.18.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11957/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11957/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11957/
Messages:
{noformat}
This message was trimmed, see log for full details
error: a/ql/src/test/results/clientpositive/spark/vectorized_case.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/vectorized_mapjoin.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/spark/vectorized_math_funcs.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/spark/vectorized_nested_mapjoin.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/spark/vectorized_ptf.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/vectorized_shufflejoin.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/spark/vectorized_string_funcs.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/special_character_in_tabnames_2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/stat_estimate_related_col.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/stats2.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/stats_empty_dyn_part.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/stats_empty_partition2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/stats_noscan_2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/stats_partial_size.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/stats_ppr_all.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/structin.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/subq.q.out: does not exist in index
error: a/ql/src/test/results/clientpositive/subq2.q.out: does not exist in index
error: a/ql/src/test/results/clientpositive/subq_where_serialization.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/subquery_alias.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/subquery_exists.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/subquery_exists_having.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/subquery_multiinsert.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/subquery_notexists.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/subquery_notexists_having.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/subquery_notin_having.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/subquery_unqual_corr_expr.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/temp_table_join1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/tez/explainanalyze_1.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/tez/explainanalyze_4.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/tez/hybridgrace_hashjoin_1.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/tez/hybridgrace_hashjoin_2.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/tez/vector_non_string_partition.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/timestamp_ints_casts.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/transform_ppr2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/truncate_column_list_bucket.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/type_widening.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/udf1.q.out: does not exist in index
error: a/ql/src/test/results/clientpositive/udf9.q.out: does not exist in index
error: a/ql/src/test/results/clientpositive/udf_10_trims.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/udf_between.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/udf_case_column_pruning.q.out: does
not exist in index
error: a/ql/
[jira] [Commented] (HIVE-19882) Fix QTestUtil session lifecycle
[
https://issues.apache.org/jira/browse/HIVE-19882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518819#comment-16518819
]
Hive QA commented on HIVE-19882:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928351/HIVE-19882.07.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11956/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11956/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11956/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12928351/HIVE-19882.07.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928351 - PreCommit-HIVE-Build
> Fix QTestUtil session lifecycle
> ---
>
> Key: HIVE-19882
> URL: https://issues.apache.org/jira/browse/HIVE-19882
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19882.01.patch, HIVE-19882.02.patch,
> HIVE-19882.03.patch, HIVE-19882.04.patch, HIVE-19882.05.patch,
> HIVE-19882.06.patch, HIVE-19882.07.patch
>
>
> there are a number of strange come and go failing tests; it was always
> strange to me that qtestutil cleans up at some questionable points - this
> seems to be leading to executing some commands with the previous qfiles
> session...
> ideally the session/etc should start/reused in {{before}}
> and it should be closed in {{after}}
> seems like configuration is handled probably incorrectly; saving the conf
> after initialization - and restoring it for a new session should ensure
> consistency
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518817#comment-16518817
]
Hive QA commented on HIVE-18725:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928345/HIVE-18725.07.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14538 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=257)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11955/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11955/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11955/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928345 - PreCommit-HIVE-Build
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch,
> HIVE-18725.06.patch, HIVE-18725.07.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HIVE-19953) query9.q fails
[ https://issues.apache.org/jira/browse/HIVE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom resolved HIVE-19953. --- Resolution: Fixed > query9.q fails > --- > > Key: HIVE-19953 > URL: https://issues.apache.org/jira/browse/HIVE-19953 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518795#comment-16518795
]
Hive QA commented on HIVE-18725:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 166 unchanged - 5
fixed = 166 total (was 171) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11955/dev-support/hive-personality.sh
|
| git revision | master / aea5908 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11955/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch,
> HIVE-18725.06.patch, HIVE-18725.07.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19946) VectorizedRowBatchCtx.recordIdColumnVector cannot be shared between different JVMs
[
https://issues.apache.org/jira/browse/HIVE-19946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518780#comment-16518780
]
Hive QA commented on HIVE-19946:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928343/HIVE-19946.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14537 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=257)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query64]
(batchId=257)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11954/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11954/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11954/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928343 - PreCommit-HIVE-Build
> VectorizedRowBatchCtx.recordIdColumnVector cannot be shared between different
> JVMs
> --
>
> Key: HIVE-19946
> URL: https://issues.apache.org/jira/browse/HIVE-19946
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-19946.1.patch
>
>
> VectorizedRowBatchCtx.recordIdColumnVector was used temporarily to pass
> record id column, which is virtual, between a reducer and a mapper. However,
> when the reducer and the mapper are not in a same JVM, it makes incorrect
> results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19943) Header values keep showing up in result sets
[
https://issues.apache.org/jira/browse/HIVE-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518776#comment-16518776
]
shaoxiaowei commented on HIVE-19943:
If it's HDInsight, I'm sorry, I can't solve this problem.
> Header values keep showing up in result sets
>
>
> Key: HIVE-19943
> URL: https://issues.apache.org/jira/browse/HIVE-19943
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.1.0
> Environment: Hdinsight Hive interactivequerry
> [Components|https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-component-versioning#hadoop-components-available-with-different-hdinsight-versions]
>Reporter: Liam De Lee
>Priority: Major
>
> We are using the tblproperties ("skip.header.line.count"="1") when creating
> an external table.
> When we do a select * from table we get it back as expected without the
> header present in the result set.
> However when we do for instance a count(1) we get the header back in this
> count (tested with a select * from table and paste it in notepad to find the
> amount of rows)
> If we also do this with a select distinct(column) from table we also get the
> header as a distinct value.
> file structure:
> ||_TESTING_TYPE||
> |adf|
> |hyg|
> |abc|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19943) Header values keep showing up in result sets
[
https://issues.apache.org/jira/browse/HIVE-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shaoxiaowei reassigned HIVE-19943:
--
Assignee: (was: shaoxiaowei)
Ask for help
> Header values keep showing up in result sets
>
>
> Key: HIVE-19943
> URL: https://issues.apache.org/jira/browse/HIVE-19943
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.1.0
> Environment: Hdinsight Hive interactivequerry
> [Components|https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-component-versioning#hadoop-components-available-with-different-hdinsight-versions]
>Reporter: Liam De Lee
>Priority: Major
>
> We are using the tblproperties ("skip.header.line.count"="1") when creating
> an external table.
> When we do a select * from table we get it back as expected without the
> header present in the result set.
> However when we do for instance a count(1) we get the header back in this
> count (tested with a select * from table and paste it in notepad to find the
> amount of rows)
> If we also do this with a select distinct(column) from table we also get the
> header as a distinct value.
> file structure:
> ||_TESTING_TYPE||
> |adf|
> |hyg|
> |abc|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19946) VectorizedRowBatchCtx.recordIdColumnVector cannot be shared between different JVMs
[
https://issues.apache.org/jira/browse/HIVE-19946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518752#comment-16518752
]
Hive QA commented on HIVE-19946:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} ql: The patch generated 0 new + 166 unchanged - 2
fixed = 166 total (was 168) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11954/dev-support/hive-personality.sh
|
| git revision | master / aea5908 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11954/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> VectorizedRowBatchCtx.recordIdColumnVector cannot be shared between different
> JVMs
> --
>
> Key: HIVE-19946
> URL: https://issues.apache.org/jira/browse/HIVE-19946
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-19946.1.patch
>
>
> VectorizedRowBatchCtx.recordIdColumnVector was used temporarily to pass
> record id column, which is virtual, between a reducer and a mapper. However,
> when the reducer and the mapper are not in a same JVM, it makes incorrect
> results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19931) stats_nonpart.q test run shows possibly wrong results.
[ https://issues.apache.org/jira/browse/HIVE-19931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19931: -- Status: Patch Available (was: Reopened) > stats_nonpart.q test run shows possibly wrong results. > -- > > Key: HIVE-19931 > URL: https://issues.apache.org/jira/browse/HIVE-19931 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19931.01.master-txnstats.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19931) stats_nonpart.q test run shows possibly wrong results.
[ https://issues.apache.org/jira/browse/HIVE-19931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518748#comment-16518748 ] Steve Yeom commented on HIVE-19931: --- Hi [~sershe] this patch includes NPE fix, query9 fix, and the last comments from the reviews. Thanks, Steve. > stats_nonpart.q test run shows possibly wrong results. > -- > > Key: HIVE-19931 > URL: https://issues.apache.org/jira/browse/HIVE-19931 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19931.01.master-txnstats.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19931) stats_nonpart.q test run shows possibly wrong results.
[ https://issues.apache.org/jira/browse/HIVE-19931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19931: -- Attachment: HIVE-19931.01.master-txnstats.patch > stats_nonpart.q test run shows possibly wrong results. > -- > > Key: HIVE-19931 > URL: https://issues.apache.org/jira/browse/HIVE-19931 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19931.01.master-txnstats.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HIVE-19931) stats_nonpart.q test run shows possibly wrong results.
[ https://issues.apache.org/jira/browse/HIVE-19931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom reopened HIVE-19931: --- This is 19931. > stats_nonpart.q test run shows possibly wrong results. > -- > > Key: HIVE-19931 > URL: https://issues.apache.org/jira/browse/HIVE-19931 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.3.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19671) Distribute by rand() can lead to data inconsistency
[
https://issues.apache.org/jira/browse/HIVE-19671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518745#comment-16518745
]
Xuefu Zhang commented on HIVE-19671:
Based on your analysis, it seems that random(see) depends on a deterministic
order of the data. Reading HDFS may guarantee the order, but probably not every
data source has such guaranty. Also, a map or reduce logic may also generate
nondeterministic order. Having said that, it appears to me that any
partitioning that depends on a deterministic ordering of the data is doomed,
include rand() and rand(seed). This is rather a user problem, for which I'm not
sure if Hive needs to do anything. We may document this as a general bad
practice, but blocking solves rand() problemI but doesn't help other similar
problems. I suggest we leave it to user to solve the problem. Thoughts?
> Distribute by rand() can lead to data inconsistency
> ---
>
> Key: HIVE-19671
> URL: https://issues.apache.org/jira/browse/HIVE-19671
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
>
> Noticed the following queries can give different results:
> {code}
> select count(*) from tbl;
> select count(*) from (select * from tbl distribute by rand()) a;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19954) TestTxnCommands2#testNonAcidToAcidConversion1 fails
[ https://issues.apache.org/jira/browse/HIVE-19954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518744#comment-16518744 ] Steve Yeom commented on HIVE-19954: --- For the case of conversion from non-acid to acid, the TBLS has no writeIdList, which caused NPE. I have a patch to add null checking in ObjectStore.isCurrentStatsValidForTheQuery() and tested the patch. This will be merged to the patch of HIVE-19931. > TestTxnCommands2#testNonAcidToAcidConversion1 fails > --- > > Key: HIVE-19954 > URL: https://issues.apache.org/jira/browse/HIVE-19954 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19954) TestTxnCommands2#testNonAcidToAcidConversion1 fails
[ https://issues.apache.org/jira/browse/HIVE-19954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom resolved HIVE-19954. --- Resolution: Fixed > TestTxnCommands2#testNonAcidToAcidConversion1 fails > --- > > Key: HIVE-19954 > URL: https://issues.apache.org/jira/browse/HIVE-19954 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19954) TestTxnCommands2#testNonAcidToAcidConversion1 fails
[ https://issues.apache.org/jira/browse/HIVE-19954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom reassigned HIVE-19954: - Assignee: Steve Yeom > TestTxnCommands2#testNonAcidToAcidConversion1 fails > --- > > Key: HIVE-19954 > URL: https://issues.apache.org/jira/browse/HIVE-19954 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19931) stats_nonpart.q test run shows possibly wrong results.
[ https://issues.apache.org/jira/browse/HIVE-19931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom resolved HIVE-19931. --- Resolution: Fixed > stats_nonpart.q test run shows possibly wrong results. > -- > > Key: HIVE-19931 > URL: https://issues.apache.org/jira/browse/HIVE-19931 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19931) stats_nonpart.q test run shows possibly wrong results.
[
https://issues.apache.org/jira/browse/HIVE-19931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518741#comment-16518741
]
Steve Yeom commented on HIVE-19931:
---
The one line incorrect code caused wrong results:
+++
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
@@ -4094,7 +4094,7 @@ public void alterTable(String catName, String dbname,
String name, Table newTabl
if (newTable.getValidWriteIdList() != null &&
TxnUtils.isTransactionalTable(newTable)) {
// Check concurrent INSERT case and set false to the flag.
-if (isCurrentStatsValidForTheQuery(oldt, newt.getTxnId(),
newt.getWriteIdList(),
+if (!isCurrentStatsValidForTheQuery(oldt, newt.getTxnId(),
newt.getWriteIdList(),
I will add this patch to the patch for HIVE-19931
> stats_nonpart.q test run shows possibly wrong results.
> --
>
> Key: HIVE-19931
> URL: https://issues.apache.org/jira/browse/HIVE-19931
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Steve Yeom
>Assignee: Steve Yeom
>Priority: Major
> Fix For: 4.0.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19922) TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky
[
https://issues.apache.org/jira/browse/HIVE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518734#comment-16518734
]
Hive QA commented on HIVE-19922:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928340/HIVE-19922.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14536 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits.testGenericUDTFOrderBySplitCount1
(batchId=243)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11953/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11953/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11953/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928340 - PreCommit-HIVE-Build
> TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky
> --
>
> Key: HIVE-19922
> URL: https://issues.apache.org/jira/browse/HIVE-19922
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19922.2.patch, HIVE-19922.3.patch, HIVE-19922.patch
>
>
> Consistently failing in the last 4 runs.
> See:
> [https://builds.apache.org/job/PreCommit-HIVE-Build/11824/testReport/org.apache.hadoop.hive.cli/TestMiniDruidKafkaCliDriver/testCliDriver_druidkafkamini_basic_/history/]
> Can not reproduce the failure locally :(
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19950) Hive ACID NOT LOCK LockComponent Correctly
[
https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-19950:
--
Component/s: Transactions
> Hive ACID NOT LOCK LockComponent Correctly
> --
>
> Key: HIVE-19950
> URL: https://issues.apache.org/jira/browse/HIVE-19950
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.3.2
>Reporter: nickSoul
>Priority: Blocker
>
> Hi,
> When using Streaming Mutation recently, I found LockComponents were not
> locked correctly by current transaction. Below is my test case:
> Step1: Begin a transaction with transactionId 126, and the transaction locks
> a table. Then hangs the transaction. The lock information were correctly
> restored in mariaDB
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +++--+---+---+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +++--+---+---+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> +++--+---+---+
> {code}
>
> Step2: Begin the other transaction with a transactionId 127 before previous
> transaction 126 finished. Transaction 127 tries to lock the same table too,
> but failed at first attempt. The lock information were correctly restored in
> mariaDB, Lock 385 was blocked by Lock 384.
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +++--+---+---+--+---+--++-+-+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +++--+---+---+--+---+--++-+-+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
> +++--+---+---+--+---+--++-+-+{code}
>
> Step3: Then transaction 127 tries to lock the table for a second retry after
> 30s with another lockId: 386, this time it successfully locked the table,
> whereas transaction 126 is still holding the lock. Lock informations in
> MetaStore DB:
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +++--+---+---+--+---+--++-+-+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +++--+---+---+--+---+--++-+-+
>
> |384| 1 |126| test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
> NULL |
> |385| 1 |127| test_acid |acid_test| NULL| w | w| NULL|384 | 1 |
> |386| 1 |127| test_acid | acid_test | NULL | a | w | 1529513069000 | NULL |
> NULL |
> +--+---+---+--+---+--++-+-+
> {code}
> {code:java}
> {code}
> After going through the code. I found it dosen't care about other
> transaction's lock on the LockComponents in second retry.
> I wonder if i use it in a wrong way, or misunderstand sth about ACID in
> hive.
>
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[
https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518713#comment-16518713
]
Hive QA commented on HIVE-17852:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 91m
43s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
37s{color} | {color:red} branch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
22s{color} | {color:red} branch/metastore cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
6s{color} | {color:red} branch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
14s{color} | {color:red} branch/standalone-metastore cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 58m
52s{color} | {color:red} root: The patch generated 1350 new + 249261 unchanged
- 1485 fixed = 250611 total (was 250746) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
50s{color} | {color:red} itests/hive-unit: The patch generated 150 new + 12394
unchanged - 157 fixed = 12544 total (was 12551) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 30m
44s{color} | {color:red} ql: The patch generated 731 new + 128471 unchanged -
839 fixed = 129202 total (was 129310) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m
17s{color} | {color:red} standalone-metastore: The patch generated 471 new +
19587 unchanged - 491 fixed = 20058 total (was 20078) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
38s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
22s{color} | {color:red} patch/metastore cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
14s{color} | {color:red} patch/standalone-metastore cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m
7s{color} | {color:red} root generated 2 new + 367 unchanged - 2 fixed = 369
total (was 369) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
55s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color
[jira] [Commented] (HIVE-19942) Hive Notification: All events for indexes should have table name
[ https://issues.apache.org/jira/browse/HIVE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518709#comment-16518709 ] Vihang Karajgaonkar commented on HIVE-19942: Patch looks good to me +1 (pending tests) > Hive Notification: All events for indexes should have table name > > > Key: HIVE-19942 > URL: https://issues.apache.org/jira/browse/HIVE-19942 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 2.3.2 >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19942.02-branch-2.patch, HIVE-19942.1.patch > > > All the events for indexes: Create Index, Alter Index, Drop Index have the > TBL_NAME as null. The TBL_NAME should be populated with the table on which > the index is created. > This makes it easier to decide whether to process the event or not without > needing to parse the json message (which is a slower process). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19942) Hive Notification: All events for indexes should have table name
[ https://issues.apache.org/jira/browse/HIVE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-19942: Attachment: HIVE-19942.02-branch-2.patch > Hive Notification: All events for indexes should have table name > > > Key: HIVE-19942 > URL: https://issues.apache.org/jira/browse/HIVE-19942 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 2.3.2 >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19942.02-branch-2.patch, HIVE-19942.1.patch > > > All the events for indexes: Create Index, Alter Index, Drop Index have the > TBL_NAME as null. The TBL_NAME should be populated with the table on which > the index is created. > This makes it easier to decide whether to process the event or not without > needing to parse the json message (which is a slower process). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19942) Hive Notification: All events for indexes should have table name
[ https://issues.apache.org/jira/browse/HIVE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-19942: Attachment: (was: HIVE-19942.2.patch) > Hive Notification: All events for indexes should have table name > > > Key: HIVE-19942 > URL: https://issues.apache.org/jira/browse/HIVE-19942 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 2.3.2 >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19942.1.patch > > > All the events for indexes: Create Index, Alter Index, Drop Index have the > TBL_NAME as null. The TBL_NAME should be populated with the table on which > the index is created. > This makes it easier to decide whether to process the event or not without > needing to parse the json message (which is a slower process). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19942) Hive Notification: All events for indexes should have table name
[ https://issues.apache.org/jira/browse/HIVE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-19942: Attachment: HIVE-19942.2.patch > Hive Notification: All events for indexes should have table name > > > Key: HIVE-19942 > URL: https://issues.apache.org/jira/browse/HIVE-19942 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 2.3.2 >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19942.1.patch, HIVE-19942.2.patch > > > All the events for indexes: Create Index, Alter Index, Drop Index have the > TBL_NAME as null. The TBL_NAME should be populated with the table on which > the index is created. > This makes it easier to decide whether to process the event or not without > needing to parse the json message (which is a slower process). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19922) TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky
[
https://issues.apache.org/jira/browse/HIVE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518704#comment-16518704
]
Hive QA commented on HIVE-19922:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11953/dev-support/hive-personality.sh
|
| git revision | master / aea5908 |
| Default Java | 1.8.0_111 |
| modules | C: itests/qtest U: itests/qtest |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11953/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky
> --
>
> Key: HIVE-19922
> URL: https://issues.apache.org/jira/browse/HIVE-19922
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19922.2.patch, HIVE-19922.3.patch, HIVE-19922.patch
>
>
> Consistently failing in the last 4 runs.
> See:
> [https://builds.apache.org/job/PreCommit-HIVE-Build/11824/testReport/org.apache.hadoop.hive.cli/TestMiniDruidKafkaCliDriver/testCliDriver_druidkafkamini_basic_/history/]
> Can not reproduce the failure locally :(
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19953) query9.q fails
[ https://issues.apache.org/jira/browse/HIVE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518687#comment-16518687 ] Steve Yeom edited comment on HIVE-19953 at 6/20/18 10:53 PM: - Unneeded TXN_ID into TAB_COL_STATS table is causing a p-commit unit test failure of query9.q. This may represent a set of test failures of "master-txnstats" branch base patch. was (Author: steveyeom2017): Unneeded TXN_ID into TAB_COL_STATS table is causing a failure. This may represent a set of test failures of "master-txnstats" branch base patch. > query9.q fails > --- > > Key: HIVE-19953 > URL: https://issues.apache.org/jira/browse/HIVE-19953 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18916) SparkClientImpl doesn't error out if spark-submit fails
[
https://issues.apache.org/jira/browse/HIVE-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518691#comment-16518691
]
Hive QA commented on HIVE-18916:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928401/HIVE-18916.5.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14540 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.testCliDriver[spark_submit_negative_executor_cores]
(batchId=259)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=243)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=243)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11952/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11952/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11952/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928401 - PreCommit-HIVE-Build
> SparkClientImpl doesn't error out if spark-submit fails
> ---
>
> Key: HIVE-18916
> URL: https://issues.apache.org/jira/browse/HIVE-18916
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18916.1.WIP.patch, HIVE-18916.2.patch,
> HIVE-18916.3.patch, HIVE-18916.4.patch, HIVE-18916.5.patch
>
>
> If {{spark-submit}} returns a non-zero exit code, {{SparkClientImpl}} will
> simply log the exit code, but won't throw an error. Eventually, the
> connection timeout will get triggered and an exception like {{Timed out
> waiting for client connection}} will be logged, which is pretty misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19928) Load Data for managed tables should set the owner of loaded files to a configurable user
[ https://issues.apache.org/jira/browse/HIVE-19928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-19928: -- Attachment: HIVE-19928.4.patch > Load Data for managed tables should set the owner of loaded files to a > configurable user > > > Key: HIVE-19928 > URL: https://issues.apache.org/jira/browse/HIVE-19928 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19928.1.patch, HIVE-19928.2.patch, > HIVE-19928.3.patch, HIVE-19928.4.patch > > > load data of managed tables should set the owner of the loaded files to a > configurable user. the default user should be hive. > If the owner of existing file is not hive, then a rename/move operation > should be replaced by copy with the copied file having hive as owner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19953) query9.q fails
[ https://issues.apache.org/jira/browse/HIVE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518690#comment-16518690 ] Steve Yeom commented on HIVE-19953: --- A patch for this jira will be merged into HIVE-19931 patch. > query9.q fails > --- > > Key: HIVE-19953 > URL: https://issues.apache.org/jira/browse/HIVE-19953 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-19953) query9.q fails
[ https://issues.apache.org/jira/browse/HIVE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-19953 started by Steve Yeom. - > query9.q fails > --- > > Key: HIVE-19953 > URL: https://issues.apache.org/jira/browse/HIVE-19953 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19953) query9.q fails
[ https://issues.apache.org/jira/browse/HIVE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom reassigned HIVE-19953: - Assignee: Steve Yeom > query9.q fails > --- > > Key: HIVE-19953 > URL: https://issues.apache.org/jira/browse/HIVE-19953 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19953) query9.q fails
[ https://issues.apache.org/jira/browse/HIVE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518687#comment-16518687 ] Steve Yeom commented on HIVE-19953: --- Unneeded TXN_ID into TAB_COL_STATS table is causing a failure. This may represent a set of test failures of "master-txnstats" branch base patch. > query9.q fails > --- > > Key: HIVE-19953 > URL: https://issues.apache.org/jira/browse/HIVE-19953 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Status: Patch Available (was: Open) Patch is based on git hash aea5908341c7b7084551216efc6c89d64b1cb06a > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, > HIVE-17751.06-standalone-metastore.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Status: Open (was: Patch Available) > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, > HIVE-17751.06-standalone-metastore.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Attachment: HIVE-17751.01.patch > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, > HIVE-17751.06-standalone-metastore.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19902) Provide Metastore micro-benchmarks
[ https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-19902: -- Attachment: (was: HIVE-17751.01.patch) > Provide Metastore micro-benchmarks > -- > > Key: HIVE-19902 > URL: https://issues.apache.org/jira/browse/HIVE-19902 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > It would be very useful to have metastore benchmarks to be able to track perf > issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19902) Provide Metastore micro-benchmarks
[ https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-19902: -- Status: Open (was: Patch Available) > Provide Metastore micro-benchmarks > -- > > Key: HIVE-19902 > URL: https://issues.apache.org/jira/browse/HIVE-19902 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > It would be very useful to have metastore benchmarks to be able to track perf > issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19902) Provide Metastore micro-benchmarks
[ https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-19902: -- Status: Patch Available (was: Open) Patch is applied on top of aea5908341c7b7084551216efc6c89d64b1cb06a > Provide Metastore micro-benchmarks > -- > > Key: HIVE-19902 > URL: https://issues.apache.org/jira/browse/HIVE-19902 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch > > > It would be very useful to have metastore benchmarks to be able to track perf > issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19902) Provide Metastore micro-benchmarks
[ https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-19902: -- Attachment: HIVE-17751.01.patch > Provide Metastore micro-benchmarks > -- > > Key: HIVE-19902 > URL: https://issues.apache.org/jira/browse/HIVE-19902 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch > > > It would be very useful to have metastore benchmarks to be able to track perf > issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18916) SparkClientImpl doesn't error out if spark-submit fails
[
https://issues.apache.org/jira/browse/HIVE-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518663#comment-16518663
]
Hive QA commented on HIVE-18916:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
54s{color} | {color:blue} itests/util in master has 55 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
45s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} spark-client in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch util passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 1 new + 14 unchanged - 3 fixed
= 15 total (was 17) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} spark-client: The patch generated 0 new + 26
unchanged - 3 fixed = 26 total (was 29) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11952/dev-support/hive-personality.sh
|
| git revision | master / aea5908 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11952/yetus/diff-checkstyle-ql.txt
|
| modules | C: itests itests/util ql spark-client U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11952/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SparkClientImpl doesn't error out if spark-submit fails
> ---
>
> Key: HIVE-18916
> URL: https://issues.apache.org/jira/browse/HIVE-18916
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18916.1.WIP.patch, HIVE-18916.2.patch,
> HIVE-18916.3.patch, HIVE-18916.4.patch, HIVE-18916.5.patch
[jira] [Commented] (HIVE-19948) HiveCli is not splitting the command by semicolon properly if quotes are inside the string
[
https://issues.apache.org/jira/browse/HIVE-19948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518633#comment-16518633
]
Sahil Takiar commented on HIVE-19948:
-
+1 LGTM
> HiveCli is not splitting the command by semicolon properly if quotes are
> inside the string
> ---
>
> Key: HIVE-19948
> URL: https://issues.apache.org/jira/browse/HIVE-19948
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-19948.1.patch
>
>
> HIVE-15297 tries to split the command by considering semicolon inside string,
> but it doesn't consider the case that quotes can also be inside string.
> For the following command {{insert into escape1 partition (ds='1', part='3')
> values ("abc' ");}}, it will fail with
> {noformat}
> 18/06/19 16:37:05 ERROR ql.Driver: FAILED: ParseException line 1:64
> extraneous input ';' expecting EOF near ''
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:64 extraneous input
> ';' expecting EOF near ''
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:220)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:74)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:67)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:606)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1686)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1633)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1628)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518627#comment-16518627
]
Alexander Kolbasov commented on HIVE-17751:
---
For
{code}
MaterializationsInvalidationCache.get().init(conf, (IHMSHandler) client);
{code}
I am going with reflection since I don't know what else can be done here.
> Separate HMS Client and HMS server into separate sub-modules
>
>
> Key: HIVE-17751
> URL: https://issues.apache.org/jira/browse/HIVE-17751
> Project: Hive
> Issue Type: Sub-task
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-17751.06-standalone-metastore.patch
>
>
> external applications which are interfacing with HMS should ideally only
> include HMSClient library instead of one big library containing server as
> well. We should ideally have a thin client library so that cross version
> support for external applications is easier. We should sub-divide the
> standalone module into possibly 3 modules (one for common classes, one for
> client classes and one for server) or 2 sub-modules (one for client and one
> for server) so that we can generate separate jars for HMS client and server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19783) Retrieve only locations in HiveMetaStore.dropPartitionsAndGetLocations
[
https://issues.apache.org/jira/browse/HIVE-19783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518624#comment-16518624
]
Hive QA commented on HIVE-19783:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928339/HIVE-19783.4.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14439 tests
executed
*Failed tests:*
{noformat}
TestBeeLineExceptionHandling - did not produce a TEST-*.xml file (likely timed
out) (batchId=193)
TestBeeLineHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestBeelineArgParsing - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestClientCommandHookFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=193)
TestHiveCli - did not produce a TEST-*.xml file (likely timed out) (batchId=193)
TestHiveSchemaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestIncrementalRows - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestReplicationScenariosAcrossInstances - did not produce a TEST-*.xml file
(likely timed out) (batchId=234)
TestShutdownHook - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
TestTableOutputFormat - did not produce a TEST-*.xml file (likely timed out)
(batchId=193)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11951/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11951/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11951/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928339 - PreCommit-HIVE-Build
> Retrieve only locations in HiveMetaStore.dropPartitionsAndGetLocations
> --
>
> Key: HIVE-19783
> URL: https://issues.apache.org/jira/browse/HIVE-19783
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19783.2.patch, HIVE-19783.4.patch, HIVE-19783.patch
>
>
> Optimize further the dropTable command.
> Currently {{HiveMetaStore.dropPartitionsAndGetLocations}} retrieves the whole
> partition object, but we need only the locations instead.
> Create a RawStore method to retrieve only the locations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19952) Error
[
https://issues.apache.org/jira/browse/HIVE-19952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518619#comment-16518619
]
Vineet Garg commented on HIVE-19952:
[~sebasan] Jira is not an appropriate place to ask questions. You should use
{{u...@hive.apache.org}} mailing list.
> Error
> -
>
> Key: HIVE-19952
> URL: https://issues.apache.org/jira/browse/HIVE-19952
> Project: Hive
> Issue Type: Bug
>Reporter: Sebastian
>Priority: Major
>
> I am joining 2 tables (hive tables in radoop rapidminer) and I am getting
> this error, someone knows why?
> * Message: HiveQL error. Message:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException:
> Can not create a Path from an ...
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19753) Strict managed tables mode in Hive
[ https://issues.apache.org/jira/browse/HIVE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19753: -- Attachment: HIVE-19753.5.patch > Strict managed tables mode in Hive > -- > > Key: HIVE-19753 > URL: https://issues.apache.org/jira/browse/HIVE-19753 > Project: Hive > Issue Type: Bug >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19753.1.patch, HIVE-19753.2.patch, > HIVE-19753.3.patch, HIVE-19753.4.patch, HIVE-19753.5.patch > > > Create a mode in Hive which enforces that all managed tables are > transactional (both full or insert-only tables allowed). Non-transactional > tables, as well as non-native tables, must be created as external tables when > this mode is enabled. > The idea would be that in strict managed tables mode all of the data written > to managed tables would have been done through Hive. > The mode would be enabled using config setting hive.strict.managed.tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)
[ https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19951: Status: Patch Available (was: Open) > Vectorization: Need to disable encoded LLAP I/O for ORC when there is data > type conversion (Schema Evolution) > -- > > Key: HIVE-19951 > URL: https://issues.apache.org/jira/browse/HIVE-19951 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19951.01.patch > > > Currently, reading encoded ORC data does not support data type conversion. > So, encoded reading and cache populating needs to be disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)
[ https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19951: Attachment: HIVE-19951.01.patch > Vectorization: Need to disable encoded LLAP I/O for ORC when there is data > type conversion (Schema Evolution) > -- > > Key: HIVE-19951 > URL: https://issues.apache.org/jira/browse/HIVE-19951 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19951.01.patch > > > Currently, reading encoded ORC data does not support data type conversion. > So, encoded reading and cache populating needs to be disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19948) HiveCli is not splitting the command by semicolon properly if quotes are inside the string
[
https://issues.apache.org/jira/browse/HIVE-19948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518596#comment-16518596
]
Aihua Xu commented on HIVE-19948:
-
[~stakiar] Can you help take a look?
> HiveCli is not splitting the command by semicolon properly if quotes are
> inside the string
> ---
>
> Key: HIVE-19948
> URL: https://issues.apache.org/jira/browse/HIVE-19948
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-19948.1.patch
>
>
> HIVE-15297 tries to split the command by considering semicolon inside string,
> but it doesn't consider the case that quotes can also be inside string.
> For the following command {{insert into escape1 partition (ds='1', part='3')
> values ("abc' ");}}, it will fail with
> {noformat}
> 18/06/19 16:37:05 ERROR ql.Driver: FAILED: ParseException line 1:64
> extraneous input ';' expecting EOF near ''
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:64 extraneous input
> ';' expecting EOF near ''
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:220)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:74)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:67)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:606)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1686)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1633)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1628)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19880) Repl Load to return recoverable vs non-recoverable error codes
[ https://issues.apache.org/jira/browse/HIVE-19880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19880: Fix Version/s: 3.1.0 > Repl Load to return recoverable vs non-recoverable error codes > -- > > Key: HIVE-19880 > URL: https://issues.apache.org/jira/browse/HIVE-19880 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19880.01.patch, HIVE-19880.04-branch-3.patch, > HIVE-19880.04.patch > > > To enable bootstrap of large databases, application has to have the ability > to keep retrying the bootstrap load till it encounters a fatal error. The > ability to identify if an error is fatal or not will be decided by hive and > communication of the same will happen to application via error codes. > So there should be different error codes for recoverable vs non-recoverable > failures which should be propagated to application as part of running the > repl load command. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19880) Repl Load to return recoverable vs non-recoverable error codes
[ https://issues.apache.org/jira/browse/HIVE-19880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518589#comment-16518589 ] Sankar Hariappan commented on HIVE-19880: - Test failures are irrelevant and failing in previous builds too. 04-branch-3.patch is committed to branch-3. > Repl Load to return recoverable vs non-recoverable error codes > -- > > Key: HIVE-19880 > URL: https://issues.apache.org/jira/browse/HIVE-19880 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19880.01.patch, HIVE-19880.04-branch-3.patch, > HIVE-19880.04.patch > > > To enable bootstrap of large databases, application has to have the ability > to keep retrying the bootstrap load till it encounters a fatal error. The > ability to identify if an error is fatal or not will be decided by hive and > communication of the same will happen to application via error codes. > So there should be different error codes for recoverable vs non-recoverable > failures which should be propagated to application as part of running the > repl load command. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19880) Repl Load to return recoverable vs non-recoverable error codes
[ https://issues.apache.org/jira/browse/HIVE-19880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19880: Resolution: Fixed Status: Resolved (was: Patch Available) > Repl Load to return recoverable vs non-recoverable error codes > -- > > Key: HIVE-19880 > URL: https://issues.apache.org/jira/browse/HIVE-19880 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19880.01.patch, HIVE-19880.04-branch-3.patch, > HIVE-19880.04.patch > > > To enable bootstrap of large databases, application has to have the ability > to keep retrying the bootstrap load till it encounters a fatal error. The > ability to identify if an error is fatal or not will be decided by hive and > communication of the same will happen to application via error codes. > So there should be different error codes for recoverable vs non-recoverable > failures which should be propagated to application as part of running the > repl load command. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[ https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518588#comment-16518588 ] Sankar Hariappan commented on HIVE-19725: - Test failures are irrelevant to the patch and failing in previous builds too. 07-branch-3.patch is committed to branch-3. > Add ability to dump non-native tables in replication metadata dump > -- > > Key: HIVE-19725 > URL: https://issues.apache.org/jira/browse/HIVE-19725 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: Repl, pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19725.01.patch, HIVE-19725.02.patch, > HIVE-19725.03.patch, HIVE-19725.04.patch, HIVE-19725.05.patch, > HIVE-19725.06-branch-3.patch, HIVE-19725.07-branch-3.patch, > HIVE-19725.07.patch > > > if hive.repl.dump.metadata.only is set to true, allow dumping non native > tables also. > Data dump for non-native tables should never be allowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[ https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19725: Resolution: Fixed Status: Resolved (was: Patch Available) > Add ability to dump non-native tables in replication metadata dump > -- > > Key: HIVE-19725 > URL: https://issues.apache.org/jira/browse/HIVE-19725 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: Repl, pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19725.01.patch, HIVE-19725.02.patch, > HIVE-19725.03.patch, HIVE-19725.04.patch, HIVE-19725.05.patch, > HIVE-19725.06-branch-3.patch, HIVE-19725.07-branch-3.patch, > HIVE-19725.07.patch > > > if hive.repl.dump.metadata.only is set to true, allow dumping non native > tables also. > Data dump for non-native tables should never be allowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[ https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19725: Fix Version/s: 3.1.0 > Add ability to dump non-native tables in replication metadata dump > -- > > Key: HIVE-19725 > URL: https://issues.apache.org/jira/browse/HIVE-19725 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: Repl, pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19725.01.patch, HIVE-19725.02.patch, > HIVE-19725.03.patch, HIVE-19725.04.patch, HIVE-19725.05.patch, > HIVE-19725.06-branch-3.patch, HIVE-19725.07-branch-3.patch, > HIVE-19725.07.patch > > > if hive.repl.dump.metadata.only is set to true, allow dumping non native > tables also. > Data dump for non-native tables should never be allowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-19931) stats_nonpart.q test run shows possibly wrong results.
[ https://issues.apache.org/jira/browse/HIVE-19931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-19931 started by Steve Yeom. - > stats_nonpart.q test run shows possibly wrong results. > -- > > Key: HIVE-19931 > URL: https://issues.apache.org/jira/browse/HIVE-19931 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19783) Retrieve only locations in HiveMetaStore.dropPartitionsAndGetLocations
[
https://issues.apache.org/jira/browse/HIVE-19783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518587#comment-16518587
]
Hive QA commented on HIVE-19783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} standalone-metastore in master has 227 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
23s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
30s{color} | {color:red} standalone-metastore: The patch generated 2 new + 1020
unchanged - 1 fixed = 1022 total (was 1021) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} standalone-metastore generated 1 new + 227 unchanged -
0 fixed = 228 total (was 227) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
8s{color} | {color:red} standalone-metastore generated 1 new + 54 unchanged - 0
fixed = 55 total (was 54) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore |
| |
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.dropPartitionsAndGetLocations(RawStore,
String, String, String, Path, boolean) makes inefficient use of keySet
iterator instead of entrySet iterator At HiveMetaStore.java:boolean) makes
inefficient use of keySet iterator instead of entrySet iterator At
HiveMetaStore.java:[line 2597] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11951/dev-support/hive-personality.sh
|
| git revision | master / aea5908 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11951/yetus/patch-mvninstall-itests_hcatalog-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11951/yetus/diff-checkstyle-standalone-metastore.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11951/yetus/new-findbugs-standalone-metastore.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11951/yetus/diff-javadoc-javadoc-standalone-metastore.txt
|
| modules | C: itests/hcatalog-unit standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11951/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Retrieve only locations in HiveMetaStore.dropPartitionsAndGetLocations
> --
>
> Key: HIVE-19783
> URL: https://issues.apache.org/jira/browse/HI
[jira] [Updated] (HIVE-19948) HiveCli is not splitting the command by semicolon properly if quotes are inside the string
[
https://issues.apache.org/jira/browse/HIVE-19948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-19948:
Status: Patch Available (was: Open)
patch-1: Handle both single quote and double quote when we are splitting by
semicolon.
> HiveCli is not splitting the command by semicolon properly if quotes are
> inside the string
> ---
>
> Key: HIVE-19948
> URL: https://issues.apache.org/jira/browse/HIVE-19948
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-19948.1.patch
>
>
> HIVE-15297 tries to split the command by considering semicolon inside string,
> but it doesn't consider the case that quotes can also be inside string.
> For the following command {{insert into escape1 partition (ds='1', part='3')
> values ("abc' ");}}, it will fail with
> {noformat}
> 18/06/19 16:37:05 ERROR ql.Driver: FAILED: ParseException line 1:64
> extraneous input ';' expecting EOF near ''
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:64 extraneous input
> ';' expecting EOF near ''
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:220)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:74)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:67)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:606)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1686)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1633)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1628)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19948) HiveCli is not splitting the command by semicolon properly if quotes are inside the string
[
https://issues.apache.org/jira/browse/HIVE-19948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-19948:
Attachment: HIVE-19948.1.patch
> HiveCli is not splitting the command by semicolon properly if quotes are
> inside the string
> ---
>
> Key: HIVE-19948
> URL: https://issues.apache.org/jira/browse/HIVE-19948
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-19948.1.patch
>
>
> HIVE-15297 tries to split the command by considering semicolon inside string,
> but it doesn't consider the case that quotes can also be inside string.
> For the following command {{insert into escape1 partition (ds='1', part='3')
> values ("abc' ");}}, it will fail with
> {noformat}
> 18/06/19 16:37:05 ERROR ql.Driver: FAILED: ParseException line 1:64
> extraneous input ';' expecting EOF near ''
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:64 extraneous input
> ';' expecting EOF near ''
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:220)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:74)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:67)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:606)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1686)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1633)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1628)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)
[ https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-19951: --- > Vectorization: Need to disable encoded LLAP I/O for ORC when there is data > type conversion (Schema Evolution) > -- > > Key: HIVE-19951 > URL: https://issues.apache.org/jira/browse/HIVE-19951 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > Currently, reading encoded ORC data does not support data type conversion. > So, encoded reading and cache populating needs to be disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[
https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518553#comment-16518553
]
Hive QA commented on HIVE-17852:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928323/HIVE-17852.10.patch
{color:green}SUCCESS:{color} +1 due to 34 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14513 tests
executed
*Failed tests:*
{noformat}
TestReplicationScenariosAcrossInstances - did not produce a TEST-*.xml file
(likely timed out) (batchId=234)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=257)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=315)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11950/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11950/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11950/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928323 - PreCommit-HIVE-Build
> remove support for list bucketing "stored as directories" in 3.0
>
>
> Key: HIVE-17852
> URL: https://issues.apache.org/jira/browse/HIVE-17852
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch,
> HIVE-17852.03.patch, HIVE-17852.04.patch, HIVE-17852.05.patch,
> HIVE-17852.06.patch, HIVE-17852.07.patch, HIVE-17852.08.patch,
> HIVE-17852.09.patch, HIVE-17852.10.patch
>
>
> From the email thread:
> 1) LB, when stored as directories, adds a lot of low-level complexity to Hive
> tables that has to be accounted for in many places in the code where the
> files are written or modified - from FSOP to ACID/replication/export.
> 2) While working on some FSOP code I noticed that some of that logic is
> broken - e.g. the duplicate file removal from tasks, a pretty fundamental
> correctness feature in Hive, may be broken. LB also doesn’t appear to be
> compatible with e.g. regular bucketing.
> 3) The feature hasn’t seen development activity in a while; it also doesn’t
> appear to be used a lot.
> Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing
> we remove it.
> (2) also suggested that, if needed, it might be easier to implement similar
> functionality by adding some flexibility to partitions (which LB directories
> look like anyway); that would also keep the logic on a higher level of
> abstraction (split generation, partition pruning) as opposed to many
> low-level places like FSOP, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19933) ALTER TABLE DROP PARTITION - Partition Not Found
[
https://issues.apache.org/jira/browse/HIVE-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518549#comment-16518549
]
Naveen Gangam commented on HIVE-19933:
--
[~belugabehr] This appears to be an optimization for commands where there are
multiple partition specs in the drop command.
{code}
0: jdbc:hive2://localhost:1> set hive.exec.drop.ignorenonexistent;
++
| set |
++
| hive.exec.drop.ignorenonexistent=true |
++
0: jdbc:hive2://localhost:1> ALTER TABLE web_logs DROP PARTITION
(`date`='xyz');
0: jdbc:hive2://localhost:1> ALTER TABLE web_logs DROP if exists PARTITION
(`date`='xyz');
0: jdbc:hive2://localhost:1> set hive.exec.drop.ignorenonexistent=false;
0: jdbc:hive2://localhost:1> set hive.exec.drop.ignorenonexistent;
+-+
| set |
+-+
| hive.exec.drop.ignorenonexistent=false |
+-+
0: jdbc:hive2://localhost:1> ALTER TABLE web_logs DROP if exists PARTITION
(`date`='xyz');
0: jdbc:hive2://localhost:1> ALTER TABLE web_logs DROP PARTITION
(`date`='xyz');
Error: Error while compiling statement: FAILED: SemanticException [Error
10006]: Partition not found (date = 'xyz') (state=42000,code=10006)
0: jdbc:hive2://localhost:1> ALTER TABLE web_logs ADD PARTITION
(`date`='date1', `date`='date2');
0: jdbc:hive2://localhost:1> ALTER TABLE web_logs DROP PARTITION
(`date`='xyz', `date`='date1', `date`='date2');
Error: Error while compiling statement: FAILED: SemanticException [Error
10006]: Partition not found (((date = 'xyz') and (date = 'date1')) and (date =
'date2')) (state=42000,code=10006)
0: jdbc:hive2://localhost:1> show partitions web_logs;
+-+
| partition |
+-+
| date=date1 |
| date=date2 |
+-+
{code}
So when there are multiple partitions specs, not finding one of parts listed
will prevent it from processing other partitions in the command. But the output
error message is controlled by a hive config property.
The error message does not clearly point out what partition is missing. What
should the behavior be when
1) if exists is specified and one or more partitions do not exist.
2) without "if exists" clause, when one or more partitions do not exist.
Thanks
> ALTER TABLE DROP PARTITION - Partition Not Found
>
>
> Key: HIVE-19933
> URL: https://issues.apache.org/jira/browse/HIVE-19933
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 1.2.2
>Reporter: BELUGA BEHR
>Assignee: Naveen Gangam
>Priority: Major
>
> {code:sql}
> ALTER TABLE web_logsz DROP PARTITION (`date`='xyz')
> -- SemanticException [Error 10001]: Table not found web_logsz
> ALTER TABLE web_logs DROP PARTITION (`date`='xyz')
> -- Success.
> {code}
> There is no 'xyz' partition for the 'date' column. To make this more
> consistent, the query should fail if the user tries to drop a partition that
> does not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19883) QTestUtil: initDataset() can be affected by the settings of the previous test
[
https://issues.apache.org/jira/browse/HIVE-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-19883:
--
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Looks like HIVE-19882 takes care of this and other issues.
> QTestUtil: initDataset() can be affected by the settings of the previous test
> -
>
> Key: HIVE-19883
> URL: https://issues.apache.org/jira/browse/HIVE-19883
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19883.1.patch, HIVE-19883.2.patch
>
>
> Tried creating a test that set
> metastore.create.as.acid/hive.create.as.insert.only, and I found that the
> built-in table default.src was being created as an insert-only transactional
> table, which will cause errors in other tests that do not set the TxnManager
> to one that supports transactional tables.
> It appears that initDataset() uses the old CliDriver that was used for the
> previous test, which has any settings used during that test:
> {noformat}
> java.lang.Exception: Creating src
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4926)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:428)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2659)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2311)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1982)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1683)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1677)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:218)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.ql.QTestUtil.initDataset(QTestUtil.java:1277)
> [classes/:?]
> at
> org.apache.hadoop.hive.ql.QTestUtil.initDataSetForTest(QTestUtil.java:1259)
> [classes/:?]
> at org.apache.hadoop.hive.ql.QTestUtil.cliInit(QTestUtil.java:1328)
> [classes/:?]
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:176)
> [classes/:?]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> [classes/:?]
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
> [test-classes/:?]
> {noformat}
> A new CliDriver is created for the new test, but only after we've created the
> dataset tables for the next test (see the line numbers for
> QTestUtil.cliInit() in both stack traces).
> {noformat}
> CliSessionState(SessionState).getConf() line: 317
> CliDriver.() line: 110
> QTestUtil.cliInit(File, boolean) line: 1360
> CoreCliDriver.runTest(String, String, String) line: 176
> CoreCliDriver(CliAdapter).runTest(String, File) line: 104
> TestMiniLlapLocalCliDriver.testCliDriver() line: 59
> {noformat}
> I think fix is to move the creation of the new CliDriver higher up in
> QTestUtil.cliInit(), before we call initDataset().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15976) Support CURRENT_CATALOG and CURRENT_SCHEMA
[
https://issues.apache.org/jira/browse/HIVE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518495#comment-16518495
]
Hive QA commented on HIVE-15976:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928313/HIVE-15976.07.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14538 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=257)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11949/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11949/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11949/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928313 - PreCommit-HIVE-Build
> Support CURRENT_CATALOG and CURRENT_SCHEMA
> --
>
> Key: HIVE-15976
> URL: https://issues.apache.org/jira/browse/HIVE-15976
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Carter Shanklin
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-15976.01.patch, HIVE-15976.02.patch,
> HIVE-15976.03.patch, HIVE-15976.04.patch, HIVE-15976.05.patch,
> HIVE-15976.06.patch, HIVE-15976.07.patch
>
>
> Support these keywords for querying the current catalog and schema. SQL
> reference: section 6.4
> *oracle*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR2(1)); -- ok
> SELECT CURRENT_SCHEMA FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> SELECT CURRENT_SCHEMA() FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> *postgres*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- error: syntax error at or
> near "CURRENT_SCHEMA"
> SELECT CURRENT_SCHEMA; -- ok, "public"
> SELECT CURRENT_SCHEMA(); -- ok, "public"
> *mysql*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- ok
> SELECT CURRENT_SCHEMA; -- error, Unknown column 'CURRENT_SCHEMA' in 'field
> list'
> SELECT CURRENT_SCHEMA(); -- error, FUNCTION db_9_e28e6f.CURRENT_SCHEMA does
> not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19882) Fix QTestUtil session lifecycle
[
https://issues.apache.org/jira/browse/HIVE-19882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518494#comment-16518494
]
Jason Dere commented on HIVE-19882:
---
+1
> Fix QTestUtil session lifecycle
> ---
>
> Key: HIVE-19882
> URL: https://issues.apache.org/jira/browse/HIVE-19882
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19882.01.patch, HIVE-19882.02.patch,
> HIVE-19882.03.patch, HIVE-19882.04.patch, HIVE-19882.05.patch,
> HIVE-19882.06.patch, HIVE-19882.07.patch
>
>
> there are a number of strange come and go failing tests; it was always
> strange to me that qtestutil cleans up at some questionable points - this
> seems to be leading to executing some commands with the previous qfiles
> session...
> ideally the session/etc should start/reused in {{before}}
> and it should be closed in {{after}}
> seems like configuration is handled probably incorrectly; saving the conf
> after initialization - and restoring it for a new session should ensure
> consistency
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518479#comment-16518479 ] Steve Yeom commented on HIVE-19532: --- Regarding the test failures on patch 04. 1. org.apache.hadoop.hive.cli: 198 failures 1.1 Golden: by enabling stats-using queries on acid tables w/ accurate table stats org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan] org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4] 1.2 Looks like unrelated to the patch org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query9] > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch, > HIVE-19532.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19892) Disable query results cache for for HiveServer2 doAs=true
[ https://issues.apache.org/jira/browse/HIVE-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19892: -- Attachment: HIVE-19892.3.patch > Disable query results cache for for HiveServer2 doAs=true > - > > Key: HIVE-19892 > URL: https://issues.apache.org/jira/browse/HIVE-19892 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19892.1.patch, HIVE-19892.2.patch, > HIVE-19892.3.patch > > > If running HS2 with doAs=true, the temp query results directory will have > ownership/permissions based on the doAs user. A subsequent query running as a > different user may not be able to access this query results directory. > Results caching will have to be disabled in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19883) QTestUtil: initDataset() can be affected by the settings of the previous test
[
https://issues.apache.org/jira/browse/HIVE-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-19883:
--
Attachment: HIVE-19883.2.patch
> QTestUtil: initDataset() can be affected by the settings of the previous test
> -
>
> Key: HIVE-19883
> URL: https://issues.apache.org/jira/browse/HIVE-19883
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19883.1.patch, HIVE-19883.2.patch
>
>
> Tried creating a test that set
> metastore.create.as.acid/hive.create.as.insert.only, and I found that the
> built-in table default.src was being created as an insert-only transactional
> table, which will cause errors in other tests that do not set the TxnManager
> to one that supports transactional tables.
> It appears that initDataset() uses the old CliDriver that was used for the
> previous test, which has any settings used during that test:
> {noformat}
> java.lang.Exception: Creating src
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4926)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:428)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2659)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2311)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1982)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1683)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1677)
> [hive-exec-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:218)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> [hive-cli-4.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.ql.QTestUtil.initDataset(QTestUtil.java:1277)
> [classes/:?]
> at
> org.apache.hadoop.hive.ql.QTestUtil.initDataSetForTest(QTestUtil.java:1259)
> [classes/:?]
> at org.apache.hadoop.hive.ql.QTestUtil.cliInit(QTestUtil.java:1328)
> [classes/:?]
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:176)
> [classes/:?]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> [classes/:?]
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
> [test-classes/:?]
> {noformat}
> A new CliDriver is created for the new test, but only after we've created the
> dataset tables for the next test (see the line numbers for
> QTestUtil.cliInit() in both stack traces).
> {noformat}
> CliSessionState(SessionState).getConf() line: 317
> CliDriver.() line: 110
> QTestUtil.cliInit(File, boolean) line: 1360
> CoreCliDriver.runTest(String, String, String) line: 176
> CoreCliDriver(CliAdapter).runTest(String, File) line: 104
> TestMiniLlapLocalCliDriver.testCliDriver() line: 59
> {noformat}
> I think fix is to move the creation of the new CliDriver higher up in
> QTestUtil.cliInit(), before we call initDataset().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15976) Support CURRENT_CATALOG and CURRENT_SCHEMA
[
https://issues.apache.org/jira/browse/HIVE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518429#comment-16518429
]
Hive QA commented on HIVE-15976:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 5 new + 72 unchanged - 0 fixed
= 77 total (was 72) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11949/dev-support/hive-personality.sh
|
| git revision | master / aea5908 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11949/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11949/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CURRENT_CATALOG and CURRENT_SCHEMA
> --
>
> Key: HIVE-15976
> URL: https://issues.apache.org/jira/browse/HIVE-15976
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Carter Shanklin
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-15976.01.patch, HIVE-15976.02.patch,
> HIVE-15976.03.patch, HIVE-15976.04.patch, HIVE-15976.05.patch,
> HIVE-15976.06.patch, HIVE-15976.07.patch
>
>
> Support these keywords for querying the current catalog and schema. SQL
> reference: section 6.4
> *oracle*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR2(1)); -- ok
> SELECT CURRENT_SCHEMA FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> SELECT CURRENT_SCHEMA() FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> *postgres*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- error: syntax error at or
> near "CURRENT_SCHEMA"
> SELECT CURRENT_SCHEMA; -- ok, "public"
> SELECT CURRENT_SCHEMA(); -- ok, "public"
> *mysql*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- ok
> SELECT CURRENT_SCHEMA; -- error, Unknown column 'CURRENT_SCHEMA' in 'field
> list'
> SELECT CURRENT_SCHEMA(); -- error, FUNCTION db_9_e28e6f.CURRENT_SCHEMA does
> not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19902) Provide Metastore micro-benchmarks
[ https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518428#comment-16518428 ] Alan Gates commented on HIVE-19902: --- I think it's fine to put it someplace else for now, as long as you're willing to sign up to move it as part of or after HIVE-17751. > Provide Metastore micro-benchmarks > -- > > Key: HIVE-19902 > URL: https://issues.apache.org/jira/browse/HIVE-19902 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > It would be very useful to have metastore benchmarks to be able to track perf > issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19891) inserting into external tables with custom partition directories may cause data loss
[
https://issues.apache.org/jira/browse/HIVE-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518423#comment-16518423
]
Sergey Shelukhin commented on HIVE-19891:
-
Updated the test to not use the local path
> inserting into external tables with custom partition directories may cause
> data loss
>
>
> Key: HIVE-19891
> URL: https://issues.apache.org/jira/browse/HIVE-19891
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19891.01.patch, HIVE-19891.02.patch,
> HIVE-19891.patch
>
>
> tbl1 is just used as a prop to create data, could be an existing directory
> for an external table.
> Due to weird behavior of LoadTableDesc (some ancient code for overriding old
> partition path), custom partition path is overwritten after the query and the
> data in it ceases being a part of the table (can be seen in desc formatted
> output with masking commented out in QTestUtil)
> This affects branch-1 too, so it's pretty old.
> {noformat}drop table tbl1;
> CREATE TABLE tbl1 (index int, value int ) PARTITIONED BY ( created_date
> string );
> insert into tbl1 partition(created_date='2018-02-01') VALUES (2, 2);
> CREATE external TABLE tbl2 (index int, value int ) PARTITIONED BY (
> created_date string );
> ALTER TABLE tbl2 ADD PARTITION(created_date='2018-02-01');
> ALTER TABLE tbl2 PARTITION(created_date='2018-02-01') SET LOCATION
> 'file:/Users/sergey/git/hivegit/itests/qtest/target/warehouse/tbl1/created_date=2018-02-01';
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> insert into tbl2 partition(created_date='2018-02-01') VALUES (1, 1);
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19891) inserting into external tables with custom partition directories may cause data loss
[
https://issues.apache.org/jira/browse/HIVE-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-19891:
Attachment: HIVE-19891.02.patch
> inserting into external tables with custom partition directories may cause
> data loss
>
>
> Key: HIVE-19891
> URL: https://issues.apache.org/jira/browse/HIVE-19891
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19891.01.patch, HIVE-19891.02.patch,
> HIVE-19891.patch
>
>
> tbl1 is just used as a prop to create data, could be an existing directory
> for an external table.
> Due to weird behavior of LoadTableDesc (some ancient code for overriding old
> partition path), custom partition path is overwritten after the query and the
> data in it ceases being a part of the table (can be seen in desc formatted
> output with masking commented out in QTestUtil)
> This affects branch-1 too, so it's pretty old.
> {noformat}drop table tbl1;
> CREATE TABLE tbl1 (index int, value int ) PARTITIONED BY ( created_date
> string );
> insert into tbl1 partition(created_date='2018-02-01') VALUES (2, 2);
> CREATE external TABLE tbl2 (index int, value int ) PARTITIONED BY (
> created_date string );
> ALTER TABLE tbl2 ADD PARTITION(created_date='2018-02-01');
> ALTER TABLE tbl2 PARTITION(created_date='2018-02-01') SET LOCATION
> 'file:/Users/sergey/git/hivegit/itests/qtest/target/warehouse/tbl1/created_date=2018-02-01';
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> insert into tbl2 partition(created_date='2018-02-01') VALUES (1, 1);
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19928) Load Data for managed tables should set the owner of loaded files to a configurable user
[ https://issues.apache.org/jira/browse/HIVE-19928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518419#comment-16518419 ] Deepak Jaiswal commented on HIVE-19928: --- Ran this test locally more than 10 times and passes all the time. > Load Data for managed tables should set the owner of loaded files to a > configurable user > > > Key: HIVE-19928 > URL: https://issues.apache.org/jira/browse/HIVE-19928 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19928.1.patch, HIVE-19928.2.patch, > HIVE-19928.3.patch > > > load data of managed tables should set the owner of the loaded files to a > configurable user. the default user should be hive. > If the owner of existing file is not hive, then a rename/move operation > should be replaced by copy with the copied file having hive as owner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19950) Hive ACID NOT LOCK LockComponent Correctly
[
https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nickSoul updated HIVE-19950:
Priority: Blocker (was: Major)
> Hive ACID NOT LOCK LockComponent Correctly
> --
>
> Key: HIVE-19950
> URL: https://issues.apache.org/jira/browse/HIVE-19950
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: nickSoul
>Priority: Blocker
>
> Hi,
> When using Streaming Mutation recently, I found LockComponents were not
> locked correctly by current transaction. Below is my test case:
> Step1: Begin a transaction with transactionId 126, and the transaction locks
> a table. Then hangs the transaction. The lock information were correctly
> restored in mariaDB
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +++--+---+---+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +++--+---+---+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> +++--+---+---+
> {code}
>
> Step2: Begin the other transaction with a transactionId 127 before previous
> transaction 126 finished. Transaction 127 tries to lock the same table too,
> but failed at first attempt. The lock information were correctly restored in
> mariaDB, Lock 385 was blocked by Lock 384.
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +++--+---+---+--+---+--++-+-+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +++--+---+---+--+---+--++-+-+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
> +++--+---+---+--+---+--++-+-+{code}
>
> Step3: Then transaction 127 tries to lock the table for a second retry after
> 30s with another lockId: 386, this time it successfully locked the table,
> whereas transaction 126 is still holding the lock. Lock informations in
> MetaStore DB:
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +++--+---+---+--+---+--++-+-+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +++--+---+---+--+---+--++-+-+
>
> |384| 1 |126| test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
> NULL |
> |385| 1 |127| test_acid |acid_test| NULL| w | w| NULL|384 | 1 |
> |386| 1 |127| test_acid | acid_test | NULL | a | w | 1529513069000 | NULL |
> NULL |
> +--+---+---+--+---+--++-+-+
> {code}
> {code:java}
> {code}
> After going through the code. I found it dosen't care about other
> transaction's lock on the LockComponents in second retry.
> I wonder if i use it in a wrong way, or misunderstand sth about ACID in
> hive.
>
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19950) Hive ACID NOT LOCK LockComponent Correctly
[
https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nickSoul updated HIVE-19950:
Description:
Hi,
When using Streaming Mutation recently, I found LockComponents were not
locked correctly by current transaction. Below is my test case:
Step1: Begin a transaction with transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
+++--+---+---+
{code}
Step2: Begin the other transaction with a transactionId 127 before previous
transaction 126 finished. Transaction 127 tries to lock the same table too, but
failed at first attempt. The lock information were correctly restored in
mariaDB, Lock 385 was blocked by Lock 384.
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+--+---+--++-+-+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+--+---+--++-+-+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
| 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
+++--+---+---+--+---+--++-+-+{code}
Step3: Then transaction 127 tries to lock the table for a second retry after
30s with another lockId: 386, this time it successfully locked the table,
whereas transaction 126 is still holding the lock. Lock informations in
MetaStore DB:
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+--+---+--++-+-+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+--+---+--++-+-+
|384| 1 |126| test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
|385| 1 |127| test_acid |acid_test| NULL| w | w| NULL|384 | 1 |
|386| 1 |127| test_acid | acid_test | NULL | a | w | 1529513069000 | NULL |
NULL |
+--+---+---+--+---+--++-+-+
{code}
{code:java}
{code}
After going through the code. I found it dosen't care about other
transaction's lock on the LockComponents in second retry.
I wonder if i use it in a wrong way, or misunderstand sth about ACID in
hive.
Thanks
was:
Hi,
When using Streaming Mutation recently, I found LockComponents were not
locked correctly by current transaction. Below is my test case:
Step1: Begin a transaction with transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | H
[jira] [Commented] (HIVE-19889) Wrong results due to PPD of non deterministic functions with CBO
[
https://issues.apache.org/jira/browse/HIVE-19889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518405#comment-16518405
]
Janaki Lahorani commented on HIVE-19889:
Thanks [~ngangam]. The failed test is not related to this patch. HIVE-19922
tracks the failed test as a flaky test.
> Wrong results due to PPD of non deterministic functions with CBO
>
>
> Key: HIVE-19889
> URL: https://issues.apache.org/jira/browse/HIVE-19889
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19889.1.patch, HIVE-19889.2.patch
>
>
> The following query can give wrong results when CBO is on:
> {code}
> select * from (
> select part1,randum123
> from (SELECT *, cast(rand() as double) AS randum123 FROM testA where
> part1='CA' and part2 = 'ABC') a
> where randum123 <= 0.5) s where s.randum123 > 0.25 limit 20;
> The plan of the query is as follows:
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: testa
> Statistics: Num rows: 2 Data size: 4580 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((rand() <= 0.5D) and (rand() > 0.25D)) (type:
> boolean)
> Statistics: Num rows: 1 Data size: 2290 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: 'CA' (type: string), rand() (type: double)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 1 Data size: 2290 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 20
> Statistics: Num rows: 1 Data size: 2290 Basic stats:
> COMPLETE Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 1 Data size: 2290 Basic stats:
> COMPLETE Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde:
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> Stage: Stage-0
> Fetch Operator
> limit: 20
> Processor Tree:
> ListSink
> {code}
> The relevant part in the plan is the filter:
> {code}
> Filter Operator
> predicate: ((rand() <= 0.5D) and (rand() > 0.25D)) (type:
> boolean)
> {code}
> The predicates randum123 <= 0.5 and s.randum123 > 0.25 were pushed down. And
> randum123 was resolved to rand(). This is bad because it will result in
> invocation of rand() two times and rand() UDF is non-deterministic. Both the
> rand calls can generate values that can satisfy the predicates independently,
> but not together, whereas the original intention of the query is to give
> results when rand falls between 0.25 and 0.5.
> A sample result:
> {code}
> CA0.9191984370369802
> CA0.397933021566812
> {code}
> where the condition was not satisfied.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19891) inserting into external tables with custom partition directories may cause data loss
[
https://issues.apache.org/jira/browse/HIVE-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518404#comment-16518404
]
Sergey Shelukhin commented on HIVE-19891:
-
Oh, the test includes local path, that's why it's only passing locally.
> inserting into external tables with custom partition directories may cause
> data loss
>
>
> Key: HIVE-19891
> URL: https://issues.apache.org/jira/browse/HIVE-19891
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19891.01.patch, HIVE-19891.patch
>
>
> tbl1 is just used as a prop to create data, could be an existing directory
> for an external table.
> Due to weird behavior of LoadTableDesc (some ancient code for overriding old
> partition path), custom partition path is overwritten after the query and the
> data in it ceases being a part of the table (can be seen in desc formatted
> output with masking commented out in QTestUtil)
> This affects branch-1 too, so it's pretty old.
> {noformat}drop table tbl1;
> CREATE TABLE tbl1 (index int, value int ) PARTITIONED BY ( created_date
> string );
> insert into tbl1 partition(created_date='2018-02-01') VALUES (2, 2);
> CREATE external TABLE tbl2 (index int, value int ) PARTITIONED BY (
> created_date string );
> ALTER TABLE tbl2 ADD PARTITION(created_date='2018-02-01');
> ALTER TABLE tbl2 PARTITION(created_date='2018-02-01') SET LOCATION
> 'file:/Users/sergey/git/hivegit/itests/qtest/target/warehouse/tbl1/created_date=2018-02-01';
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> insert into tbl2 partition(created_date='2018-02-01') VALUES (1, 1);
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19950) Hive ACID NOT LOCK LockComponent Correctly
[
https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nickSoul updated HIVE-19950:
Description:
Hi,
When using Streaming Mutation recently, I found LockComponents were not locked
correctly by current transaction. Below is my test case:
Step1: Begin a transaction with transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
+++--+---+---+
{code}
Step2: Begin the other transaction with a transactionId 127 before previous
transaction 126 finished. Transaction 127 tries to lock the same table too, but
failed at first attempt. The lock information were correctly restored in
mariaDB, Lock 385 was blocked by Lock 384.
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+--+---+--++-+-+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+--+---+--++-+-+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
| 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
+++--+---+---+--+---+--++-+-+{code}
Step3: Then transaction 127 tries to lock the table for a second retry after
30s with another lockId: 386, this time it successfully locked the table,
whereas transaction 126 is still holding the lock. Lock informations in
MetaStore DB:
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+--+---+--++-+-+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+--+---+--++-+-+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
| NULL | | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 |
1 | | 386 | 1 | 127 | test_acid | acid_test | NULL | a | w | 1529513069000 |
NULL | NULL |
+++--+---+---+--+---+--++-+-+
{code}
{code:java}
{code}
After going through the code. I found it dosen't care about other
transaction's lock on the LockComponents in second retry.
I wonder if i use it in a wrong way, or misunderstand sth about ACID in
hive.
Thanks
was:
Hi,
When using Streaming Mutation recently, I found LockComponents were not locked
correctly by current transaction. Below is my test case:
Step1: Begin a transaction with transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+
| HL_LOCK_EXT_ID | HL_LOC
[jira] [Updated] (HIVE-19950) Hive ACID NOT LOCK LockComponent Correctly
[
https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nickSoul updated HIVE-19950:
Description:
Hi,
When using Streaming Mutation recently, I found LockComponents were not
locked correctly by current transaction. Below is my test case:
Step1: Begin a transaction with transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
+++--+---+---+
{code}
Step2: Begin the other transaction with a transactionId 127 before previous
transaction 126 finished. Transaction 127 tries to lock the same table too, but
failed at first attempt. The lock information were correctly restored in
mariaDB, Lock 385 was blocked by Lock 384.
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+--+---+--++-+-+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+--+---+--++-+-+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
| 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
+++--+---+---+--+---+--++-+-+{code}
Step3: Then transaction 127 tries to lock the table for a second retry after
30s with another lockId: 386, this time it successfully locked the table,
whereas transaction 126 is still holding the lock. Lock informations in
MetaStore DB:
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+--+---+--++-+-+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+++--+---+---+--+---+--++-+-+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
| NULL | | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 |
1 | | 386 | 1 | 127 | test_acid | acid_test | NULL | a | w | 1529513069000 |
NULL | NULL |
+++--+---+---+--+---+--++-+-+
{code}
{code:java}
{code}
After going through the code. I found it dosen't care about other
transaction's lock on the LockComponents in second retry.
I wonder if i use it in a wrong way, or misunderstand sth about ACID in
hive.
Thanks
was:
Hi,
When using Streaming Mutation recently, I found LockComponents were not locked
correctly by current transaction. Below is my test case:
Step1: Begin a transaction with transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+++--+---+---+
| HL_LOCK_EXT_ID | HL
[jira] [Commented] (HIVE-19897) Add more tests for parallel compilation
[ https://issues.apache.org/jira/browse/HIVE-19897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518401#comment-16518401 ] Aihua Xu commented on HIVE-19897: - Looks good to me. +1. > Add more tests for parallel compilation > > > Key: HIVE-19897 > URL: https://issues.apache.org/jira/browse/HIVE-19897 > Project: Hive > Issue Type: Test > Components: HiveServer2 >Reporter: Yongzhi Chen >Assignee: Yongzhi Chen >Priority: Minor > Attachments: HIVE-19897.1.patch, HIVE-19897.3.patch > > > The two parallel compilation tests in > org.apache.hive.jdbc.TestJdbcWithMiniHS2 do not real cover the case of > queries compile concurrently from different connections. No sure it is on > purpose or by mistake. Add more tests to cover the case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
