[jira] [Commented] (HIVE-19358) CBO decorrelation logic should generate Hive operators
[ https://issues.apache.org/jira/browse/HIVE-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499314#comment-16499314 ] Ashutosh Chauhan commented on HIVE-19358: - +1 > CBO decorrelation logic should generate Hive operators > -- > > Key: HIVE-19358 > URL: https://issues.apache.org/jira/browse/HIVE-19358 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19358.01.patch, HIVE-19358.02.patch, > HIVE-19358.03.patch, HIVE-19358.04.patch, HIVE-19358.05.patch, > HIVE-19358.patch, fix.patch > > > Decorrelation logic may generate logical instances of the operators in the > plan (e.g., LogicalFilter instead of HiveFilter). This leads to errors while > costing the tree in the Volcano planner (used in MV rewriting), since logical > operators do not have a cost associated to them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19695) Year Month Day extraction functions need to add an implicit cast for column that are String types
[
https://issues.apache.org/jira/browse/HIVE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499312#comment-16499312
]
Hive QA commented on HIVE-19695:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
30s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11461/dev-support/hive-personality.sh
|
| git revision | master / 3bccc4e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11461/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Year Month Day extraction functions need to add an implicit cast for column
> that are String types
> -
>
> Key: HIVE-19695
> URL: https://issues.apache.org/jira/browse/HIVE-19695
> Project: Hive
> Issue Type: Bug
> Components: Druid integration, Query Planning
>Affects Versions: 3.0.0
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19695.1.patch, HIVE-19695.patch, HIVE-19695.patch
>
>
> To avoid surprising/wrong results, Hive Query plan shall add an explicit cast
> over non date/timestamp column type when user try to extract Year/Month/Hour
> etc..
> This is an example of misleading results.
> {code}
> create table test_base_table(`timecolumn` timestamp, `date_c` string,
> `timestamp_c` string, `metric_c` double);
> insert into test_base_table values ('2015-03-08 00:00:00', '2015-03-10',
> '2015-03-08 00:00:00', 5.0);
> CREATE TABLE druid_test_table
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.segment.granularity" = "DAY")
> AS select
> cast(`timecolumn` as timestamp with local time zone) as `__time`, `date_c`,
> `timestamp_c`, `metric_c` FROM test_base_table;
> select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table;
> {code}
> will return the following wrong results:
> {code}
> PREHOOK: query: select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c),
[jira] [Updated] (HIVE-18792) Allow standard compliant syntax for insert on partitioned tables
[
https://issues.apache.org/jira/browse/HIVE-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18792:
Fix Version/s: (was: 4.0.0)
3.1.0
> Allow standard compliant syntax for insert on partitioned tables
>
>
> Key: HIVE-18792
> URL: https://issues.apache.org/jira/browse/HIVE-18792
> Project: Hive
> Issue Type: Improvement
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-18792.1.patch, HIVE-18792.2.patch,
> HIVE-18792.3.patch
>
>
> Following works:
> {code}
> create table t1 (a int, b int, c int);
> create table t2 (a int, b int, c int) partitioned by (d int);
> insert into t1 values (1,2,3);
> insert into t1 (c, b, a) values (1,2,3);
> insert into t1 (a,b) values (1,2);
> {code}
> For partitioned tables it should work similarly but doesn't. All of
> following fails:
> {code}
> insert into t2 values (1,2,3,4);
> insert into t2 (a, b, c, d) values (1,2,3,4);
> insert into t2 (c,d) values (1,2);
> insert into t2 (a,b) values (1,2);
> {code}
> All of above should work. Also note following works:
> {code}
> insert into t2 partition(d) values (1,2,3,4);
> insert into t2 partition(d=4) values (1,2,3);
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19462) Fix mapping for char_length function to enable pushdown to Druid.
[ https://issues.apache.org/jira/browse/HIVE-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19462: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master and branch-3. Thanks, Slim! > Fix mapping for char_length function to enable pushdown to Druid. > -- > > Key: HIVE-19462 > URL: https://issues.apache.org/jira/browse/HIVE-19462 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19462.2.patch, HIVE-19462.3.patch, > HIVE-19462.4.patch, HIVE-19462.patch > > > currently char_length is not push down to Druid because of missing mapping > form/to calcite > This patch will add this mapping. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19718) Adding partitions in bulk also fetches table for each partition
[
https://issues.apache.org/jira/browse/HIVE-19718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499305#comment-16499305
]
Hive QA commented on HIVE-19718:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925884/HIVE-19718.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 535 failed/errored test(s), 9559 tests
executed
*Failed tests:*
{noformat}
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=213)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=213)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=213)
TestE2EScenarios - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=213)
TestHCatLoaderEncryption - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestHCatStorerMulti - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestHCatStorerWrapper - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=213)
TestMetaStoreConnectionUrlHook - did not produce a TEST-*.xml file (likely
timed out) (batchId=213)
TestMetaStoreEndFunctionListener - did not produce a TEST-*.xml file (likely
timed out) (batchId=213)
TestMetastoreConf - did not produce a TEST-*.xml file (likely timed out)
(batchId=213)
TestMetrics - did not produce a TEST-*.xml file (likely timed out) (batchId=213)
TestOrcHCatLoader - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestParquetHCatStorer - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestPartitionNameWhitelistValidation - did not produce a TEST-*.xml file
(likely timed out) (batchId=213)
TestPigHCatUtil - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestRCFileHCatLoader - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestSchemaToolForMetastore - did not produce a TEST-*.xml file (likely timed
out) (batchId=213)
TestStats - did not produce a TEST-*.xml file (likely timed out) (batchId=213)
TestThriftCLIServiceWithHttp - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestUtil - did not produce a TEST-*.xml file (likely timed out) (batchId=198)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_10]
(batchId=255)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[create_database]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[create_like]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_blobstore]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_local]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_warehouse]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_local_to_blobstore]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_local]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_local_to_blobstore]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_nonstd_partitions_loc]
(batchId=258)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[rcfile_nonstd_partitions_loc]
(batchId=258)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=18)
[jira] [Commented] (HIVE-19773) CBO exception while running queries with tables that are not present in materialized views
[ https://issues.apache.org/jira/browse/HIVE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499302#comment-16499302 ] Ashutosh Chauhan commented on HIVE-19773: - +1 > CBO exception while running queries with tables that are not present in > materialized views > -- > > Key: HIVE-19773 > URL: https://issues.apache.org/jira/browse/HIVE-19773 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.1.0, 4.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Attachments: HIVE-19773.patch > > > When we obtain the valid list of write ids, some tables in the materialized > views may not be present in the list because they are not present in the > query, which leads to exceptions (hidden in logs) when we try to load the > materialized views in the planner, as we need to verify whether they are > outdated or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19751) create submodule of hive-upgrade-acid for preUpgrade and postUpgrade
[ https://issues.apache.org/jira/browse/HIVE-19751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499299#comment-16499299 ] Ashutosh Chauhan commented on HIVE-19751: - +1 > create submodule of hive-upgrade-acid for preUpgrade and postUpgrade > > > Key: HIVE-19751 > URL: https://issues.apache.org/jira/browse/HIVE-19751 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-19751.02.patch > > > Basically need to produce 2 separate jars: 1 for pre-upgrade step that can be > compiled/unit tested with 2.x jars and another can be compiled/tested with > 3.x jars. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19718) Adding partitions in bulk also fetches table for each partition
[
https://issues.apache.org/jira/browse/HIVE-19718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499298#comment-16499298
]
Hive QA commented on HIVE-19718:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
42s{color} | {color:blue} standalone-metastore in master has 214 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 3 new + 962
unchanged - 3 fixed = 965 total (was 965) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s{color} | {color:green} standalone-metastore generated 0 new + 213 unchanged
- 1 fixed = 213 total (was 214) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11459/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11459/yetus/patch-mvninstall-itests_hcatalog-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11459/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: itests/hcatalog-unit standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11459/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Adding partitions in bulk also fetches table for each partition
> ---
>
> Key: HIVE-19718
> URL: https://issues.apache.org/jira/browse/HIVE-19718
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
> Environment: Looking at {{convertToMPart}}:
> {code:Java}
> private MPartition convertToMPart(Partition part, boolean useTableCD)
> throws InvalidObjectException, MetaException {
> MTable mt = getMTable(part.getCatName(), part.getDbName(),
> part.getTableName());
> ...
> {code}
> So what we have as a result is that we fetch table for every partition where
> it should be done just once.
>Reporter: Alexander Kolbasov
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19718.2.patch, HIVE-19718.patch
>
>
> The ObjectStore.addPartitions() method
[jira] [Commented] (HIVE-19768) Utility to convert tables to conform to Hive strict managed tables mode
[ https://issues.apache.org/jira/browse/HIVE-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499297#comment-16499297 ] Ashutosh Chauhan commented on HIVE-19768: - +1 > Utility to convert tables to conform to Hive strict managed tables mode > --- > > Key: HIVE-19768 > URL: https://issues.apache.org/jira/browse/HIVE-19768 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19768.1.patch > > > Create a utility that can check existing hive tables and convert them if > necessary to conform to strict managed tables mode. > - Managed non-transactional ORC tables will be converted to full > transactional tables > - Managed non-transactional tables of other types will be converted to > insert-only transactional tables > - Tables with non-native storage/schema will be converted to external tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[
https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499295#comment-16499295
]
Hive QA commented on HIVE-19725:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925881/HIVE-19725.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14443 tests
executed
*Failed tests:*
{noformat}
TestHs2HooksWithMiniKdc - did not produce a TEST-*.xml file (likely timed out)
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testShouldDumpMetaDataForNonNativeTableIfSetMeataDataOnly
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11458/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11458/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11458/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925881 - PreCommit-HIVE-Build
> Add ability to dump non-native tables in replication metadata dump
> --
>
> Key: HIVE-19725
> URL: https://issues.apache.org/jira/browse/HIVE-19725
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.0.0, 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: Repl, pull-request-available
> Fix For: 3.1.0, 3.0.1, 4.0.0
>
> Attachments: HIVE-19725.01.patch, HIVE-19725.02.patch
>
>
> if hive.repl.dump.metadata.only is set to true, allow dumping non native
> tables also.
> Data dump for non-native tables should never be allowed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[
https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499285#comment-16499285
]
Hive QA commented on HIVE-19725:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
31s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 52
unchanged - 0 fixed = 53 total (was 52) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11458/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11458/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11458/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add ability to dump non-native tables in replication metadata dump
> --
>
> Key: HIVE-19725
> URL: https://issues.apache.org/jira/browse/HIVE-19725
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.0.0, 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: Repl, pull-request-available
> Fix For: 3.1.0, 3.0.1, 4.0.0
>
> Attachments: HIVE-19725.01.patch, HIVE-19725.02.patch
>
>
> if hive.repl.dump.metadata.only is set to true, allow dumping non native
> tables also.
> Data dump for non-native tables should never be allowed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16505) Support "unknown" boolean truth value
[
https://issues.apache.org/jira/browse/HIVE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499275#comment-16499275
]
Hive QA commented on HIVE-16505:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925899/HIVE-16505.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 1 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input31] (batchId=63)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11457/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11457/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11457/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925899 - PreCommit-HIVE-Build
> Support "unknown" boolean truth value
> -
>
> Key: HIVE-16505
> URL: https://issues.apache.org/jira/browse/HIVE-16505
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-16505.01.patch, HIVE-16505.02.patch
>
>
> according to the standard, boolean truth value might be:
> {{TRUE|FALSE|UNKNOWN}}.
> similar queries to the following should be supported:
> {code:java}
> select 1 where null is unknown;
> select 1 where (select cast(null as boolean) ) is unknown;
> {code}
> "unknown" behaves similarily to null. {{(null=null) is null}}
>
> "All boolean values and SQL truth values are comparable and all are
> assignable to a site of type boolean. The value True is greater than the
> value False, and any comparison involving the null value or an Unknown truth
> value will return an Unknown result. The values True and False may be
> assigned to any site having a boolean data type; assignment of Unknown, or
> the null value, is subject to the nullability characteristic of the target."
>
> *Truth table for the AND boolean operator*
> AND True False Unknown
> True True False Unknown
> False False False False
> Unknown Unknown False Unknown
> *Truth table for the OR boolean operator*
> OR True False Unknown
> True True True True
> False True False Unknown
> Unknown True Unknown Unknown
> *Truth table for the IS boolean operator*
> IS TRUE FALSE UNKNOWN
> True True False False
> False False True False
> Unknown False False True
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499262#comment-16499262 ] Thejas M Nair commented on HIVE-19776: -- 3.patch- Minor change to not log error/exception when interruptedexception is received by the thrift server thread during shutdown > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19776.1.patch, HIVE-19776.2.patch, > HIVE-19776.3.patch > > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19776: - Status: Patch Available (was: Open) > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19776.1.patch, HIVE-19776.2.patch, > HIVE-19776.3.patch > > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19776: - Attachment: HIVE-19776.3.patch > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19776.1.patch, HIVE-19776.2.patch, > HIVE-19776.3.patch > > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16505) Support "unknown" boolean truth value
[
https://issues.apache.org/jira/browse/HIVE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499259#comment-16499259
]
Hive QA commented on HIVE-16505:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11457/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11457/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support "unknown" boolean truth value
> -
>
> Key: HIVE-16505
> URL: https://issues.apache.org/jira/browse/HIVE-16505
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-16505.01.patch, HIVE-16505.02.patch
>
>
> according to the standard, boolean truth value might be:
> {{TRUE|FALSE|UNKNOWN}}.
> similar queries to the following should be supported:
> {code:java}
> select 1 where null is unknown;
> select 1 where (select cast(null as boolean) ) is unknown;
> {code}
> "unknown" behaves similarily to null. {{(null=null) is null}}
>
> "All boolean values and SQL truth values are comparable and all are
> assignable to a site of type boolean. The value True is greater than the
> value False, and any comparison involving the null value or an Unknown truth
> value will return an Unknown result. The values True and False may be
> assigned to any site having a boolean data type; assignment of Unknown, or
> the null value, is subject to the nullability characteristic of the target."
>
> *Truth table for the AND boolean operator*
> AND True False Unknown
> True True False Unknown
> False False False False
> Unknown Unknown False Unknown
> *Truth table for the OR boolean operator*
> OR True False Unknown
> True True True True
> False True False Unknown
> Unknown True Unknown Unknown
> *Truth table for the IS boolean operator*
> IS TRUE FALSE UNKNOWN
> True True False False
> False False True False
> Unknown False False True
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19645) Console info/error messages should be accessible by clients using hive server as well
[
https://issues.apache.org/jira/browse/HIVE-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499257#comment-16499257
]
Hive QA commented on HIVE-19645:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925879/HIVE-19645.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11456/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11456/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11456/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-06-03 01:40:04.435
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-11456/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-06-03 01:40:04.449
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1c5faaa HIVE-18792 : Allow standard compliant syntax for insert
on partitioned tables (Vineet Garg via Ashutosh Chauhan)
+ git clean -f -d
Removing itests/${project.basedir}/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1c5faaa HIVE-18792 : Allow standard compliant syntax for insert
on partitioned tables (Vineet Garg via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-06-03 01:40:11.555
+ rm -rf ../yetus_PreCommit-HIVE-Build-11456
+ mkdir ../yetus_PreCommit-HIVE-Build-11456
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-11456
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-11456/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java: does
not exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc1530518641419238736.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc1530518641419238736.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 40 classes.
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveLexer.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
org/apache/hadoop/hive/ql/parse/HiveLexer.g
Output file
[jira] [Commented] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[
https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499256#comment-16499256
]
Hive QA commented on HIVE-19629:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925877/HIVE-19629.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11455/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11455/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11455/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925877/HIVE-19629.7.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925877 - PreCommit-HIVE-Build
> Enable Decimal64 reader after orc version upgrade
> -
>
> Key: HIVE-19629
> URL: https://issues.apache.org/jira/browse/HIVE-19629
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19629.1.patch, HIVE-19629.2.patch,
> HIVE-19629.3.patch, HIVE-19629.4.patch, HIVE-19629.5.patch,
> HIVE-19629.6.patch, HIVE-19629.7.patch
>
>
> ORC 1.5.0 supports new fast decimal 64 reader. New VRB has to be created for
> making use of decimal 64 column vectors. Also LLAP IO will need a new reader
> to reader from long stream to decimal 64.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[
https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499255#comment-16499255
]
Hive QA commented on HIVE-19629:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925877/HIVE-19629.7.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 364 failed/errored test(s), 14411 tests
executed
*Failed tests:*
{noformat}
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=186)
[insert_overwrite_directory2.q,spark_dynamic_partition_pruning_4.q,vector_outer_join0.q,bucket4.q,orc_merge4.q,bucket5.q,infer_bucket_sort_merge.q,orc_merge_incompat1.q,root_dir_external_table.q,constprog_partitioner.q,constprog_semijoin.q,external_table_with_space_in_location_path.q,spark_constprog_dpp.q,spark_dynamic_partition_pruning_3.q,load_fs2.q,infer_bucket_sort_map_operators.q,spark_dynamic_partition_pruning_2.q,vector_inner_join.q,spark_multi_insert_parallel_orderby.q,remote_script.q]
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=187)
[scriptfile1.q,vector_outer_join5.q,file_with_header_footer.q,input16_cc.q,orc_merge2.q,reduce_deduplicate.q,schemeAuthority2.q,spark_dynamic_partition_pruning_5.q,orc_merge8.q,orc_merge_incompat2.q,infer_bucket_sort_bucketed_table.q,vector_outer_join4.q,disable_merge_for_bucketing.q,orc_merge7.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input31] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mergejoin] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_file_dump]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge11] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge5] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge6] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat1]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_schema_evolution_float]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_struct_type_vectorization]
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[type_change_test_int]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[type_change_test_int_vectorized]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_aggregate_9]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_aggregate_without_gby]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_between_columns]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_binary_join_groupby]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_bround]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_1]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_2]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_cast_constant]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_2]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_4]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_mapjoin1]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_simple]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_2]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_3]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_4]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_count]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_data_types]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_date_1]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_10_0]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_1]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_5]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_6]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_aggregate]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_cast]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_expressions]
(batchId=55)
[jira] [Commented] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[
https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499248#comment-16499248
]
Hive QA commented on HIVE-19629:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} llap-server in master has 86 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} llap-server: The patch generated 26 new + 265
unchanged - 10 fixed = 291 total (was 275) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
50s{color} | {color:red} ql: The patch generated 81 new + 1819 unchanged - 6
fixed = 1900 total (was 1825) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
44s{color} | {color:red} ql generated 2 new + 2278 unchanged - 0 fixed = 2280
total (was 2278) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatchCtx.setRowDataTypePhysicalVariations(DataTypePhysicalVariation[])
may expose internal representation by storing an externally mutable object
into VectorizedRowBatchCtx.rowDataTypePhysicalVariations At
VectorizedRowBatchCtx.java:by storing an externally mutable object into
VectorizedRowBatchCtx.rowDataTypePhysicalVariations At
VectorizedRowBatchCtx.java:[line 168] |
| | Switch statement found in
org.apache.hadoop.hive.ql.io.orc.WriterImpl.setColumn(int, ColumnVector,
ObjectInspector, Object) where default case is missing At
WriterImpl.java:ColumnVector, ObjectInspector, Object) where default case is
missing At WriterImpl.java:[lines 138-227] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11454/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11454/yetus/patch-mvninstall-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11454/yetus/diff-checkstyle-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11454/yetus/diff-checkstyle-ql.txt
|
| findbugs |
[jira] [Commented] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499241#comment-16499241 ] Daniel Dai commented on HIVE-19776: --- +1 pending test. > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19776.1.patch, HIVE-19776.2.patch > > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[
https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499235#comment-16499235
]
Hive QA commented on HIVE-18140:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925871/HIVE-19140.02wip02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11453/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11453/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11453/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925871/HIVE-19140.02wip02.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925871 - PreCommit-HIVE-Build
> Partitioned tables statistics can go wrong in basic stats mixed case
>
>
> Key: HIVE-18140
> URL: https://issues.apache.org/jira/browse/HIVE-18140
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch,
> HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02wip01.patch,
> HIVE-19140.02wip02.patch
>
>
> suppose the following scenario:
> * part1 has basic stats {{RC=10,DS=1K}}
> * all other partition has no basic stats (and a bunch of rows)
> then
> [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378]
> condition would be false; which in turn produces estimations for the whole
> partitioned table: {{RC=10,DS=1K}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19727) Fix Signature matching of table aliases
[
https://issues.apache.org/jira/browse/HIVE-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499234#comment-16499234
]
Hive QA commented on HIVE-19727:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925867/HIVE-19727.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11452/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11452/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11452/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925867/HIVE-19727.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925867 - PreCommit-HIVE-Build
> Fix Signature matching of table aliases
> ---
>
> Key: HIVE-19727
> URL: https://issues.apache.org/jira/browse/HIVE-19727
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19727.01.patch, HIVE-19727.02.patch,
> HIVE-19727.03.patch
>
>
> there is a probable problem with alias matching: "t1 as a" is matched to "t2
> as a"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
[
https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499233#comment-16499233
]
Hive QA commented on HIVE-18079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925865/HIVE-18079.20.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14471 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_0]
(batchId=176)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=242)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11451/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11451/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11451/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925865 - PreCommit-HIVE-Build
> Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator
> bit-size
>
>
> Key: HIVE-18079
> URL: https://issues.apache.org/jira/browse/HIVE-18079
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Statistics
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18079-branch-3.patch, HIVE-18079-branch-3.patch,
> HIVE-18079-branch-3.patch, HIVE-18079.1.patch, HIVE-18079.10.patch,
> HIVE-18079.11.patch, HIVE-18079.12.patch, HIVE-18079.13.patch,
> HIVE-18079.14.patch, HIVE-18079.15.patch, HIVE-18079.15.patch,
> HIVE-18079.15.patch, HIVE-18079.16.patch, HIVE-18079.17.patch,
> HIVE-18079.17.patch, HIVE-18079.18.patch, HIVE-18079.19.patch,
> HIVE-18079.2.patch, HIVE-18079.20.patch, HIVE-18079.4.patch,
> HIVE-18079.5.patch, HIVE-18079.6.patch, HIVE-18079.7.patch,
> HIVE-18079.8.patch, HIVE-18079.9.patch
>
>
> HyperLogLog can merge a 14 bit HLL into a 10 bit HLL bitset, because of its
> mathematical hash distribution & construction.
> Allow the squashing of a 14 bit HLL -> 10 bit HLL without needing a second
> scan over the data-set.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19776: - Attachment: HIVE-19776.2.patch > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19776.1.patch, HIVE-19776.2.patch > > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
[
https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499218#comment-16499218
]
Hive QA commented on HIVE-18079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
29s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
44s{color} | {color:blue} standalone-metastore in master has 214 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} standalone-metastore: The patch generated 12 new + 36
unchanged - 1 fixed = 48 total (was 37) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11451/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11451/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11451/yetus/whitespace-eol.txt
|
| modules | C: ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11451/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator
> bit-size
>
>
> Key: HIVE-18079
> URL: https://issues.apache.org/jira/browse/HIVE-18079
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Statistics
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18079-branch-3.patch, HIVE-18079-branch-3.patch,
> HIVE-18079-branch-3.patch, HIVE-18079.1.patch, HIVE-18079.10.patch,
> HIVE-18079.11.patch, HIVE-18079.12.patch, HIVE-18079.13.patch,
> HIVE-18079.14.patch, HIVE-18079.15.patch, HIVE-18079.15.patch,
> HIVE-18079.15.patch, HIVE-18079.16.patch,
[jira] [Commented] (HIVE-19096) query result cache interferes with explain analyze
[
https://issues.apache.org/jira/browse/HIVE-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499215#comment-16499215
]
Hive QA commented on HIVE-19096:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925860/HIVE-19096.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14443 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11450/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11450/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11450/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925860 - PreCommit-HIVE-Build
> query result cache interferes with explain analyze
> ---
>
> Key: HIVE-19096
> URL: https://issues.apache.org/jira/browse/HIVE-19096
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19096.1.patch, HIVE-19096.2.patch,
> HIVE-19096.3.patch
>
>
> if result cache is active; the explain analyze doesn't really return usefull
> informations; even for unseen queries the result is like this:
> {code}
> ++
> |Explain |
> ++
> | Stage-0|
> | Fetch Operator |
> | Cached Query Result:true,limit:-1 |
> ||
> ++
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19776: - Attachment: HIVE-19776.1.patch > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19776.1.patch > > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499208#comment-16499208 ] Thejas M Nair commented on HIVE-19776: -- The change is mainly to refactor the initialization of server and that of stop so that stop doesn't go ahead and try to stop before initialization thread has actually initialized the server. Also added thread.interrupt to interrupt the background thread. The blocking call to the server (server.join() ) do check for interrupts and would stop. > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19776.1.patch > > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-19776: -- Labels: pull-request-available (was: ) > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499205#comment-16499205 ] ASF GitHub Bot commented on HIVE-19776: --- GitHub user thejasmn opened a pull request: https://github.com/apache/hive/pull/363 HIVE-19776 1.patch You can merge this pull request into a Git repository by running: $ git pull https://github.com/thejasmn/hive HIVE-19776 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/363.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #363 commit 85c7505d3719415fb39707c43053fcf19f4fe838 Author: Thejas M Nair Date: 2018-06-02T19:46:08Z HIVE-19776 1.patch > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19096) query result cache interferes with explain analyze
[
https://issues.apache.org/jira/browse/HIVE-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499201#comment-16499201
]
Hive QA commented on HIVE-19096:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
34s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11450/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11450/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> query result cache interferes with explain analyze
> ---
>
> Key: HIVE-19096
> URL: https://issues.apache.org/jira/browse/HIVE-19096
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19096.1.patch, HIVE-19096.2.patch,
> HIVE-19096.3.patch
>
>
> if result cache is active; the explain analyze doesn't really return usefull
> informations; even for unseen queries the result is like this:
> {code}
> ++
> |Explain |
> ++
> | Stage-0|
> | Fetch Operator |
> | Cached Query Result:true,limit:-1 |
> ||
> ++
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19529) Vectorization: Date/Timestamp NULL issues
[
https://issues.apache.org/jira/browse/HIVE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499192#comment-16499192
]
Hive QA commented on HIVE-19529:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925937/HIVE-19529.06-branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11449/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11449/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11449/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925937/HIVE-19529.06-branch-3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925937 - PreCommit-HIVE-Build
> Vectorization: Date/Timestamp NULL issues
> -
>
> Key: HIVE-19529
> URL: https://issues.apache.org/jira/browse/HIVE-19529
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Blocker
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19529.06-branch-3.patch, HIVE-19529.06.patch
>
>
> Wrong results found for:
> date_add/date_sub
> UT areas:
> date_add/date_sub
> datediff
> to_date
> interval_year_month + interval_year_month
> interval_day_time + interval_day_time
> interval_day_time + timestamp
> timestamp + interval_day_time
> date + interval_day_time
> interval_day_time + date
> interval_year_month + date
> date + interval_year_month
> interval_year_month + interval_year_month
> timestamp + interval_year_month
> date - date
> interval_year_month - interval_year_month
> interval_day_time - interval_day_time
> timestamp - interval_day_time
> timestamp - timestamp
> date - timestamp
> timestamp - date
> date - interval_day_time
> date - interval_year_month
> timestamp - interval_year_month
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19334) Use actual file size rather than stats for fetch task optimization with external tables
[
https://issues.apache.org/jira/browse/HIVE-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499190#comment-16499190
]
Hive QA commented on HIVE-19334:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925857/HIVE-19334.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14443 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11448/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11448/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11448/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925857 - PreCommit-HIVE-Build
> Use actual file size rather than stats for fetch task optimization with
> external tables
> ---
>
> Key: HIVE-19334
> URL: https://issues.apache.org/jira/browse/HIVE-19334
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19334.1.patch, HIVE-19334.2.patch,
> HIVE-19334.3.patch, HIVE-19334.4.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19334) Use actual file size rather than stats for fetch task optimization with external tables
[
https://issues.apache.org/jira/browse/HIVE-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499177#comment-16499177
]
Hive QA commented on HIVE-19334:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11448/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11448/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use actual file size rather than stats for fetch task optimization with
> external tables
> ---
>
> Key: HIVE-19334
> URL: https://issues.apache.org/jira/browse/HIVE-19334
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19334.1.patch, HIVE-19334.2.patch,
> HIVE-19334.3.patch, HIVE-19334.4.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19332) Disable compute.query.using.stats for external table
[
https://issues.apache.org/jira/browse/HIVE-19332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499166#comment-16499166
]
Hive QA commented on HIVE-19332:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12926043/HIVE-19332.8.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 1 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input31] (batchId=63)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11447/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11447/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11447/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12926043 - PreCommit-HIVE-Build
> Disable compute.query.using.stats for external table
>
>
> Key: HIVE-19332
> URL: https://issues.apache.org/jira/browse/HIVE-19332
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19332.1.patch, HIVE-19332.2.patch,
> HIVE-19332.3.patch, HIVE-19332.4.patch, HIVE-19332.5.patch,
> HIVE-19332.6.patch, HIVE-19332.7.patch, HIVE-19332.8.patch
>
>
> Hive can use statistics to answer queries like count(*). This can be
> problematic on external tables where another tool might add files that Hive
> doesn’t know about. In that case Hive will return incorrect results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19776) HiveServer2.startHiveServer2 retries of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19776: - Summary: HiveServer2.startHiveServer2 retries of start has concurrency issues (was: HS2 retry of start has concurrency issues) > HiveServer2.startHiveServer2 retries of start has concurrency issues > > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19776) HS2 retry of start has concurrency issues
[ https://issues.apache.org/jira/browse/HIVE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair reassigned HIVE-19776: > HS2 retry of start has concurrency issues > - > > Key: HIVE-19776 > URL: https://issues.apache.org/jira/browse/HIVE-19776 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > > HS2 starts the thrift binary/http servers in background, while it proceeds to > do other setup (eg create zookeeper entries). If there is a ZK error and it > attempts to stop and start in the retry loop within > HiveServer2.startHiveServer2, the retry fails because the thrift server > doesn't get stopped if it was still getting initialized. > The thrift server initialization and stopping needs to be synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19332) Disable compute.query.using.stats for external table
[
https://issues.apache.org/jira/browse/HIVE-19332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499151#comment-16499151
]
Hive QA commented on HIVE-19332:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
34s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 2 new + 383 unchanged - 0
fixed = 385 total (was 383) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11447/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11447/yetus/diff-checkstyle-ql.txt
|
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11447/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Disable compute.query.using.stats for external table
>
>
> Key: HIVE-19332
> URL: https://issues.apache.org/jira/browse/HIVE-19332
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19332.1.patch, HIVE-19332.2.patch,
> HIVE-19332.3.patch, HIVE-19332.4.patch, HIVE-19332.5.patch,
> HIVE-19332.6.patch, HIVE-19332.7.patch, HIVE-19332.8.patch
>
>
> Hive can use statistics to answer queries like count(*). This can be
> problematic on external tables where another tool might add files that Hive
> doesn’t know about. In that case Hive will return incorrect results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19708) Repl copy retrying with cm path even if the failure is due to network issue
[
https://issues.apache.org/jira/browse/HIVE-19708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499144#comment-16499144
]
Hive QA commented on HIVE-19708:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12926093/HIVE-19708.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14443 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.createGetDrop[Remote]
(batchId=211)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11446/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11446/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11446/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12926093 - PreCommit-HIVE-Build
> Repl copy retrying with cm path even if the failure is due to network issue
> ---
>
> Key: HIVE-19708
> URL: https://issues.apache.org/jira/browse/HIVE-19708
> Project: Hive
> Issue Type: Task
> Components: Hive, HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19708.01.patch, HIVE-19708.02.patch,
> HIVE-19708.04.patch, HIVE-19708.05.patch
>
>
> * During repl load
> ** for filesystem based copying of file if the copy fails due to a
> connection error to source Name Node, we should recreate the filesystem
> object.
> ** the retry logic for local file copy should be triggered using the
> original source file path ( and not the CM root path ) since failure can be
> due to network issues between DFSClient and NN.
> * When listing files in tables / partition to include them in _files, we
> should add retry logic when failure occurs. FileSystem object here also
> should be recreated since the existing one might be in inconsistent state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19708) Repl copy retrying with cm path even if the failure is due to network issue
[
https://issues.apache.org/jira/browse/HIVE-19708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499141#comment-16499141
]
Hive QA commented on HIVE-19708:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
30s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
32s{color} | {color:red} ql: The patch generated 1 new + 10 unchanged - 0 fixed
= 11 total (was 10) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11446/dev-support/hive-personality.sh
|
| git revision | master / 1c5faaa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11446/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11446/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Repl copy retrying with cm path even if the failure is due to network issue
> ---
>
> Key: HIVE-19708
> URL: https://issues.apache.org/jira/browse/HIVE-19708
> Project: Hive
> Issue Type: Task
> Components: Hive, HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19708.01.patch, HIVE-19708.02.patch,
> HIVE-19708.04.patch, HIVE-19708.05.patch
>
>
> * During repl load
> ** for filesystem based copying of file if the copy fails due to a
> connection error to source Name Node, we should recreate the filesystem
> object.
> ** the retry logic for local file copy should be triggered using the
> original source file path ( and not the CM root path ) since failure can be
> due to network issues between DFSClient and NN.
> * When listing files in
[jira] [Updated] (HIVE-19775) Schematool should use HS2 embedded mode in privileged auth mode
[ https://issues.apache.org/jira/browse/HIVE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19775: - Attachment: HIVE-19775.1.patch > Schematool should use HS2 embedded mode in privileged auth mode > --- > > Key: HIVE-19775 > URL: https://issues.apache.org/jira/browse/HIVE-19775 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Critical > Attachments: HIVE-19775.1.patch > > > Follow up of HIVE-19389. > Authorization checks don't make sense for embedded mode and since it is not > used in that mode it leads to issues if authorization is enabled (eg, > username not set). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19775) Schematool should use HS2 embedded mode in privileged auth mode
[ https://issues.apache.org/jira/browse/HIVE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair reassigned HIVE-19775: > Schematool should use HS2 embedded mode in privileged auth mode > --- > > Key: HIVE-19775 > URL: https://issues.apache.org/jira/browse/HIVE-19775 > Project: Hive > Issue Type: Improvement >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Critical > > Follow up of HIVE-19389. > Authorization checks don't make sense for embedded mode and since it is not > used in that mode it leads to issues if authorization is enabled (eg, > username not set). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17896) TopNKey: Create a standalone vectorizable TopNKey operator
[
https://issues.apache.org/jira/browse/HIVE-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499135#comment-16499135
]
Hive QA commented on HIVE-17896:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925842/HIVE-17896.9.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14449 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input31] (batchId=63)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[topnkey]
(batchId=106)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vector_topnkey]
(batchId=106)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11445/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11445/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11445/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925842 - PreCommit-HIVE-Build
> TopNKey: Create a standalone vectorizable TopNKey operator
> --
>
> Key: HIVE-17896
> URL: https://issues.apache.org/jira/browse/HIVE-17896
> Project: Hive
> Issue Type: New Feature
> Components: Operators
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-17896.1.patch, HIVE-17896.3.patch,
> HIVE-17896.4.patch, HIVE-17896.5.patch, HIVE-17896.6.patch,
> HIVE-17896.7.patch, HIVE-17896.8.patch, HIVE-17896.9.patch
>
>
> For TPC-DS Query27, the TopN operation is delayed by the group-by - the
> group-by operator buffers up all the rows before discarding the 99% of the
> rows in the TopN Hash within the ReduceSink Operator.
> The RS TopN operator is very restrictive as it only supports doing the
> filtering on the shuffle keys, but it is better to do this before breaking
> the vectors into rows and losing the isRepeating properties.
> Adding a TopN Key operator in the physical operator tree allows the following
> to happen.
> GBY->RS(Top=1)
> can become
> TNK(1)->GBY->RS(Top=1)
> So that, the TopNKey can remove rows before they are buffered into the GBY
> and consume memory.
> Here's the equivalent implementation in Presto
> https://github.com/prestodb/presto/blob/master/presto-main/src/main/java/com/facebook/presto/operator/TopNOperator.java#L35
> Adding this as a sub-feature of GroupBy prevents further optimizations if the
> GBY is on keys "a,b,c" and the TopNKey is on just "a".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19750) Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
[ https://issues.apache.org/jira/browse/HIVE-19750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-19750: -- Status: Open (was: Patch Available) > Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full > acid > --- > > Key: HIVE-19750 > URL: https://issues.apache.org/jira/browse/HIVE-19750 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Fix For: 3.1.0 > > Attachments: HIVE-19750.01.patch, HIVE-19750.02.patch > > > Need to set this to a reasonably high value the the table. > This will reserve a range of write IDs that will be treated by the system as > committed. > This is needed so that we can assign unique ROW__IDs to each row in files > that already exist in the table. For example, if the value is initialized to > the number of files currently in the table, we can think of each file as > written by a separate transaction and thus a free to assign bucketProperty > (BucketCodec) of ROW_ID in whichever way is convenient. > it's guaranteed that all rows get unique ROW_IDs this way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19750) Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
[ https://issues.apache.org/jira/browse/HIVE-19750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-19750: -- Attachment: HIVE-19750.02.patch > Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full > acid > --- > > Key: HIVE-19750 > URL: https://issues.apache.org/jira/browse/HIVE-19750 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Fix For: 3.1.0 > > Attachments: HIVE-19750.01.patch, HIVE-19750.02.patch > > > Need to set this to a reasonably high value the the table. > This will reserve a range of write IDs that will be treated by the system as > committed. > This is needed so that we can assign unique ROW__IDs to each row in files > that already exist in the table. For example, if the value is initialized to > the number of files currently in the table, we can think of each file as > written by a separate transaction and thus a free to assign bucketProperty > (BucketCodec) of ROW_ID in whichever way is convenient. > it's guaranteed that all rows get unique ROW_IDs this way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19750) Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
[ https://issues.apache.org/jira/browse/HIVE-19750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-19750: -- Status: Patch Available (was: Open) > Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full > acid > --- > > Key: HIVE-19750 > URL: https://issues.apache.org/jira/browse/HIVE-19750 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Fix For: 3.1.0 > > Attachments: HIVE-19750.01.patch, HIVE-19750.02.patch > > > Need to set this to a reasonably high value the the table. > This will reserve a range of write IDs that will be treated by the system as > committed. > This is needed so that we can assign unique ROW__IDs to each row in files > that already exist in the table. For example, if the value is initialized to > the number of files currently in the table, we can think of each file as > written by a separate transaction and thus a free to assign bucketProperty > (BucketCodec) of ROW_ID in whichever way is convenient. > it's guaranteed that all rows get unique ROW_IDs this way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17896) TopNKey: Create a standalone vectorizable TopNKey operator
[
https://issues.apache.org/jira/browse/HIVE-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499127#comment-16499127
]
Hive QA commented on HIVE-17896:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} serde in master has 190 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
27s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 37 new + 397 unchanged - 0
fixed = 434 total (was 397) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
44s{color} | {color:red} serde generated 1 new + 190 unchanged - 0 fixed = 191
total (was 190) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
44s{color} | {color:red} ql generated 8 new + 2278 unchanged - 0 fixed = 2286
total (was 2278) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:serde |
| |
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(Object[],
ObjectInspector[], Object[], ObjectInspector[], boolean[]) negates the return
value of
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(Object,
ObjectInspector, Object, ObjectInspector) At
ObjectInspectorUtils.java:negates the return value of
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(Object,
ObjectInspector, Object, ObjectInspector) At ObjectInspectorUtils.java:[line
958] |
| FindBugs | module:ql |
| | new
org.apache.hadoop.hive.ql.exec.TopNKeyOperator$KeyWrapperComparator(ObjectInspector[],
ObjectInspector[], boolean[]) may expose internal representation by storing an
externally mutable object into
TopNKeyOperator$KeyWrapperComparator.columnSortOrderIsDesc At
TopNKeyOperator.java:expose internal representation by storing an externally
mutable object into TopNKeyOperator$KeyWrapperComparator.columnSortOrderIsDesc
At TopNKeyOperator.java:[line 71] |
| | new
org.apache.hadoop.hive.ql.exec.TopNKeyOperator$KeyWrapperComparator(ObjectInspector[],
ObjectInspector[], boolean[]) may expose internal representation by storing an
externally mutable object into
TopNKeyOperator$KeyWrapperComparator.objectInspectors1 At
TopNKeyOperator.java:expose internal representation by storing an externally
mutable object into
[jira] [Updated] (HIVE-18792) Allow standard compliant syntax for insert on partitioned tables
[
https://issues.apache.org/jira/browse/HIVE-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18792:
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Vineet!
It will be useful for branch-3 as well.
> Allow standard compliant syntax for insert on partitioned tables
>
>
> Key: HIVE-18792
> URL: https://issues.apache.org/jira/browse/HIVE-18792
> Project: Hive
> Issue Type: Improvement
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-18792.1.patch, HIVE-18792.2.patch,
> HIVE-18792.3.patch
>
>
> Following works:
> {code}
> create table t1 (a int, b int, c int);
> create table t2 (a int, b int, c int) partitioned by (d int);
> insert into t1 values (1,2,3);
> insert into t1 (c, b, a) values (1,2,3);
> insert into t1 (a,b) values (1,2);
> {code}
> For partitioned tables it should work similarly but doesn't. All of
> following fails:
> {code}
> insert into t2 values (1,2,3,4);
> insert into t2 (a, b, c, d) values (1,2,3,4);
> insert into t2 (c,d) values (1,2);
> insert into t2 (a,b) values (1,2);
> {code}
> All of above should work. Also note following works:
> {code}
> insert into t2 partition(d) values (1,2,3,4);
> insert into t2 partition(d=4) values (1,2,3);
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18792) Allow standard compliant syntax for insert on partitioned tables
[
https://issues.apache.org/jira/browse/HIVE-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499122#comment-16499122
]
Ashutosh Chauhan commented on HIVE-18792:
-
+1
> Allow standard compliant syntax for insert on partitioned tables
>
>
> Key: HIVE-18792
> URL: https://issues.apache.org/jira/browse/HIVE-18792
> Project: Hive
> Issue Type: Improvement
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18792.1.patch, HIVE-18792.2.patch,
> HIVE-18792.3.patch
>
>
> Following works:
> {code}
> create table t1 (a int, b int, c int);
> create table t2 (a int, b int, c int) partitioned by (d int);
> insert into t1 values (1,2,3);
> insert into t1 (c, b, a) values (1,2,3);
> insert into t1 (a,b) values (1,2);
> {code}
> For partitioned tables it should work similarly but doesn't. All of
> following fails:
> {code}
> insert into t2 values (1,2,3,4);
> insert into t2 (a, b, c, d) values (1,2,3,4);
> insert into t2 (c,d) values (1,2);
> insert into t2 (a,b) values (1,2);
> {code}
> All of above should work. Also note following works:
> {code}
> insert into t2 partition(d) values (1,2,3,4);
> insert into t2 partition(d=4) values (1,2,3);
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19525) Spark task logs print PLAN PATH excessive number of times
[
https://issues.apache.org/jira/browse/HIVE-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharathkrishna Guruvayoor Murali updated HIVE-19525:
Attachment: HIVE-19525.2.patch
> Spark task logs print PLAN PATH excessive number of times
> -
>
> Key: HIVE-19525
> URL: https://issues.apache.org/jira/browse/HIVE-19525
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19525.1.patch, HIVE-19525.2.patch
>
>
> A ton of logs with this {{Utilities - PLAN PATH =
> hdfs://localhost:59527/.../apache-hive/itests/qtest-spark/target/tmp/scratchdir/stakiar/6ebceb49-7a76-4159-9082-5bba44391e30/hive_2018-05-14_07-28-44_672_8205774950452575544-1/-mr-10006/bf14c0b5-a014-4ee8-8ddf-fdb7453eb0f0/map.xml}}
> Seems it print multiple times per task exception, not sure where it is coming
> from, but its too verbose. It should be changed to DEBUG level. Furthermore,
> given that we are using {{Utilities#getBaseWork}} anytime we need to access a
> {{MapWork}} or {{ReduceWork}} object, we should make the method slightly more
> efficient. Right now it borrows a {{Kryo}} from a pool and does a bunch of
> stuff to set the classloader, then it checks the cache to see if the work
> object has already been created. It should check the cache before doing any
> of that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19418) add background stats updater similar to compactor
[
https://issues.apache.org/jira/browse/HIVE-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499119#comment-16499119
]
Hive QA commented on HIVE-19418:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12926175/HIVE-19418.06.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14446 tests
executed
*Failed tests:*
{noformat}
TestActivePassiveHA - did not produce a TEST-*.xml file (likely timed out)
(batchId=242)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11444/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11444/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11444/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12926175 - PreCommit-HIVE-Build
> add background stats updater similar to compactor
> -
>
> Key: HIVE-19418
> URL: https://issues.apache.org/jira/browse/HIVE-19418
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19418.01.patch, HIVE-19418.02.patch,
> HIVE-19418.03.patch, HIVE-19418.04.patch, HIVE-19418.05.patch,
> HIVE-19418.06.patch, HIVE-19418.06.patch, HIVE-19418.patch
>
>
> There's a JIRA HIVE-19416 to add snapshot version to stats for MM/ACID tables
> to make them usable in a transaction without breaking ACID (for metadata-only
> optimization). However, stats for ACID tables can still become unusable if
> e.g. two parallel inserts run - neither sees the data written by the other,
> so after both finish, the snapshots on either set of stats won't match the
> current snapshot and the stats will be unusable.
> Additionally, for ACID and non-ACID tables alike, a lot of the stats, with
> some exceptions like numRows, cannot be aggregated (i.e. you cannot combine
> ndvs from two inserts), and for ACID even less can be aggregated (you cannot
> derive min/max if some rows are deleted but you don't scan the rest of the
> dataset).
> Therefore we will add background logic to metastore (similar to, and
> partially inside, the ACID compactor) to update stats.
> It will have 3 modes of operation.
> 1) Off.
> 2) Update only the stats that exist but are out of date (generating stats can
> be expensive, so if the user is only analyzing a subset of tables it should
> be able to only update that subset). We can simply look at existing stats and
> only analyze for the relevant partitions and columns.
> 3) On: 2 + create stats for all tables and columns missing stats.
> There will also be a table parameter to skip stats update.
> In phase 1, the process will operate outside of compactor, and run analyze
> command on the table. The analyze command will automatically save the stats
> with ACID snapshot information if needed, based on HIVE-19416, so we don't
> need to do any special state management and this will work for all table
> types. However it's also more expensive.
> In phase 2, we can explore adding stats collection during MM compaction that
> uses a temp table. If we don't have open writers during major compaction (so
> we overwrite all of the data), the temp table stats can simply be copied over
> to the main table with correct snapshot information, saving us a table scan.
> In phase 3, we can add custom stats collection logic to full ACID compactor
> that is not query based, the same way as we'd do for (2). Alternatively we
> can wait for ACID compactor to become query based and just reuse (2).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19418) add background stats updater similar to compactor
[
https://issues.apache.org/jira/browse/HIVE-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499116#comment-16499116
]
Hive QA commented on HIVE-19418:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
26s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
35s{color} | {color:blue} standalone-metastore in master has 214 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
28s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} itests/hcatalog-unit: The patch generated 1 new + 27
unchanged - 0 fixed = 28 total (was 27) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} ql: The patch generated 32 new + 120 unchanged - 0
fixed = 152 total (was 120) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} standalone-metastore: The patch generated 16 new +
1470 unchanged - 0 fixed = 1486 total (was 1470) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
41s{color} | {color:red} ql generated 4 new + 2278 unchanged - 0 fixed = 2282
total (was 2278) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
52s{color} | {color:red} standalone-metastore generated 2 new + 213 unchanged -
1 fixed = 215 total (was 214) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to writeIds in
org.apache.hadoop.hive.ql.stats.StatsUpdaterThread.processOneTable(MetaStoreUtils$FullTableName)
At
StatsUpdaterThread.java:org.apache.hadoop.hive.ql.stats.StatsUpdaterThread.processOneTable(MetaStoreUtils$FullTableName)
At StatsUpdaterThread.java:[line 221] |
| | Synchronization performed on java.util.concurrent.atomic.AtomicInteger in
org.apache.hadoop.hive.ql.stats.StatsUpdaterThread.markAnalyzeDone(StatsUpdaterThread$AnalyzeWork)
At
StatsUpdaterThread.java:org.apache.hadoop.hive.ql.stats.StatsUpdaterThread.markAnalyzeDone(StatsUpdaterThread$AnalyzeWork)
At StatsUpdaterThread.java:[line 501] |
| | Synchronization performed on java.util.concurrent.atomic.AtomicInteger in
org.apache.hadoop.hive.ql.stats.StatsUpdaterThread.waitForQueuedCommands() At
StatsUpdaterThread.java:org.apache.hadoop.hive.ql.stats.StatsUpdaterThread.waitForQueuedCommands()
At StatsUpdaterThread.java:[line 642] |
| |
[jira] [Comment Edited] (HIVE-19751) create submodule of hive-upgrade-acid for preUpgrade and postUpgrade
[ https://issues.apache.org/jira/browse/HIVE-19751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499115#comment-16499115 ] Eugene Koifman edited comment on HIVE-19751 at 6/2/18 4:07 PM: --- https://builds.apache.org/job/PreCommit-HIVE-Build/11462 was (Author: ekoifman): https://builds.apache.org/job/PreCommit-HIVE-Build/1%E2%80%8B14%E2%80%8B62 > create submodule of hive-upgrade-acid for preUpgrade and postUpgrade > > > Key: HIVE-19751 > URL: https://issues.apache.org/jira/browse/HIVE-19751 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-19751.02.patch > > > Basically need to produce 2 separate jars: 1 for pre-upgrade step that can be > compiled/unit tested with 2.x jars and another can be compiled/tested with > 3.x jars. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19751) create submodule of hive-upgrade-acid for preUpgrade and postUpgrade
[ https://issues.apache.org/jira/browse/HIVE-19751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499115#comment-16499115 ] Eugene Koifman commented on HIVE-19751: --- https://builds.apache.org/job/PreCommit-HIVE-Build/1%E2%80%8B14%E2%80%8B62 > create submodule of hive-upgrade-acid for preUpgrade and postUpgrade > > > Key: HIVE-19751 > URL: https://issues.apache.org/jira/browse/HIVE-19751 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-19751.02.patch > > > Basically need to produce 2 separate jars: 1 for pre-upgrade step that can be > compiled/unit tested with 2.x jars and another can be compiled/tested with > 3.x jars. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[
https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499108#comment-16499108
]
Hive QA commented on HIVE-17852:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925827/HIVE-17852.05.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11443/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11443/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11443/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-06-02 15:32:18.617
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-11443/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-06-02 15:32:18.620
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 4463c2b HIVE-19432 : GetTablesOperation is too slow if the hive
has too many databases and tables (Rajkumar Singh via Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 4463c2b HIVE-19432 : GetTablesOperation is too slow if the hive
has too many databases and tables (Rajkumar Singh via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-06-02 15:32:20.276
+ rm -rf ../yetus_PreCommit-HIVE-Build-11443
+ mkdir ../yetus_PreCommit-HIVE-Build-11443
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-11443
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-11443/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:22
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java' with
conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:3799: trailing whitespace.
numFiles1
/data/hiveptest/working/scratch/build.patch:3859: trailing whitespace.
numFiles1
/data/hiveptest/working/scratch/build.patch:3863: trailing whitespace.
totalSize 5293
/data/hiveptest/working/scratch/build.patch:3902: trailing whitespace.
numFiles1
/data/hiveptest/working/scratch/build.patch:3923: trailing whitespace.
numFiles1
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:22
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java' with
conflicts.
U ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
warning: squelched 46 whitespace errors
warning: 51 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-11443
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925827 - PreCommit-HIVE-Build
> remove support for list bucketing "stored as directories" in 3.0
>
>
> Key: HIVE-17852
> URL: https://issues.apache.org/jira/browse/HIVE-17852
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch,
[jira] [Commented] (HIVE-18792) Allow standard compliant syntax for insert on partitioned tables
[
https://issues.apache.org/jira/browse/HIVE-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499107#comment-16499107
]
Hive QA commented on HIVE-18792:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925820/HIVE-18792.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14443 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11441/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11441/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11441/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925820 - PreCommit-HIVE-Build
> Allow standard compliant syntax for insert on partitioned tables
>
>
> Key: HIVE-18792
> URL: https://issues.apache.org/jira/browse/HIVE-18792
> Project: Hive
> Issue Type: Improvement
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18792.1.patch, HIVE-18792.2.patch,
> HIVE-18792.3.patch
>
>
> Following works:
> {code}
> create table t1 (a int, b int, c int);
> create table t2 (a int, b int, c int) partitioned by (d int);
> insert into t1 values (1,2,3);
> insert into t1 (c, b, a) values (1,2,3);
> insert into t1 (a,b) values (1,2);
> {code}
> For partitioned tables it should work similarly but doesn't. All of
> following fails:
> {code}
> insert into t2 values (1,2,3,4);
> insert into t2 (a, b, c, d) values (1,2,3,4);
> insert into t2 (c,d) values (1,2);
> insert into t2 (a,b) values (1,2);
> {code}
> All of above should work. Also note following works:
> {code}
> insert into t2 partition(d) values (1,2,3,4);
> insert into t2 partition(d=4) values (1,2,3);
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC
[
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499098#comment-16499098
]
Hive QA commented on HIVE-12192:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} serde in master has 190 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} druid-handler in master has 13 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} hcatalog/core in master has 33 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} hcatalog/hcatalog-pig-adapter in master has 2 extant
Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
47s{color} | {color:red} branch/itests/hive-jmh cannot run convertXmlToText
from findbugs {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} llap-common in master has 71 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} llap-server in master has 86 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch storage-api passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} common: The patch generated 12 new + 466 unchanged -
16 fixed = 478 total (was 482) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
31s{color} | {color:red} serde: The patch generated 13 new + 3454 unchanged -
30 fixed = 3467 total (was 3484) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 8m
12s{color} | {color:red} root: The patch generated 418 new + 11019 unchanged -
267 fixed = 11437 total (was 11286) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} druid-handler: The patch generated 2 new + 136
unchanged - 0 fixed = 138 total (was 136) {color} |
|
[jira] [Commented] (HIVE-18792) Allow standard compliant syntax for insert on partitioned tables
[
https://issues.apache.org/jira/browse/HIVE-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499097#comment-16499097
]
Hive QA commented on HIVE-18792:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
2s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11441/dev-support/hive-personality.sh
|
| git revision | master / 4463c2b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11441/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Allow standard compliant syntax for insert on partitioned tables
>
>
> Key: HIVE-18792
> URL: https://issues.apache.org/jira/browse/HIVE-18792
> Project: Hive
> Issue Type: Improvement
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18792.1.patch, HIVE-18792.2.patch,
> HIVE-18792.3.patch
>
>
> Following works:
> {code}
> create table t1 (a int, b int, c int);
> create table t2 (a int, b int, c int) partitioned by (d int);
> insert into t1 values (1,2,3);
> insert into t1 (c, b, a) values (1,2,3);
> insert into t1 (a,b) values (1,2);
> {code}
> For partitioned tables it should work similarly but doesn't. All of
> following fails:
> {code}
> insert into t2 values (1,2,3,4);
> insert into t2 (a, b, c, d) values (1,2,3,4);
> insert into t2 (c,d) values (1,2);
> insert into t2 (a,b) values (1,2);
> {code}
> All of above should work. Also note following works:
> {code}
> insert into t2 partition(d) values (1,2,3,4);
> insert into t2 partition(d=4) values (1,2,3);
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16295) Add support for using Hadoop's S3A OutputCommitter
[
https://issues.apache.org/jira/browse/HIVE-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499091#comment-16499091
]
Hive QA commented on HIVE-16295:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925816/HIVE-16295.6.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11440/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11440/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11440/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-06-02 14:20:51.957
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-11440/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-06-02 14:20:51.965
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 4463c2b HIVE-19432 : GetTablesOperation is too slow if the hive
has too many databases and tables (Rajkumar Singh via Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 4463c2b HIVE-19432 : GetTablesOperation is too slow if the hive
has too many databases and tables (Rajkumar Singh via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-06-02 14:20:59.032
+ rm -rf ../yetus_PreCommit-HIVE-Build-11440
+ mkdir ../yetus_PreCommit-HIVE-Build-11440
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-11440
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-11440/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not
exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java:
does not exist in index
error: a/ql/pom.xml: does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredContext.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecMapper.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecReducer.java: does
not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkMapRecordHandler.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: does not
exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/plan/MoveWork.java: does not
exist in index
error: a/ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java: does
not exist in index
error: a/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveCopyFiles.java:
does not exist in index
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:18
Falling back to three-way merge...
Applied patch to
[jira] [Commented] (HIVE-19529) Vectorization: Date/Timestamp NULL issues
[
https://issues.apache.org/jira/browse/HIVE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499088#comment-16499088
]
Hive QA commented on HIVE-19529:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925937/HIVE-19529.06-branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11439/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11439/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11439/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925937/HIVE-19529.06-branch-3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925937 - PreCommit-HIVE-Build
> Vectorization: Date/Timestamp NULL issues
> -
>
> Key: HIVE-19529
> URL: https://issues.apache.org/jira/browse/HIVE-19529
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Blocker
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19529.06-branch-3.patch, HIVE-19529.06.patch
>
>
> Wrong results found for:
> date_add/date_sub
> UT areas:
> date_add/date_sub
> datediff
> to_date
> interval_year_month + interval_year_month
> interval_day_time + interval_day_time
> interval_day_time + timestamp
> timestamp + interval_day_time
> date + interval_day_time
> interval_day_time + date
> interval_year_month + date
> date + interval_year_month
> interval_year_month + interval_year_month
> timestamp + interval_year_month
> date - date
> interval_year_month - interval_year_month
> interval_day_time - interval_day_time
> timestamp - interval_day_time
> timestamp - timestamp
> date - timestamp
> timestamp - date
> date - interval_day_time
> date - interval_year_month
> timestamp - interval_year_month
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC
[
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499087#comment-16499087
]
Hive QA commented on HIVE-12192:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12926116/HIVE-12192.08.patch
{color:green}SUCCESS:{color} +1 due to 63 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 179 failed/errored test(s), 14447 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.accumulo.predicate.TestAccumuloRangeGenerator.testDateRangeConjunction
(batchId=191)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[udf_unix_timestamp]
(batchId=255)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cast_on_constant]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[confirm_initial_tbl_stats]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_timestamptz2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_join_preds]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input31] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_alt] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_timestamp2]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_timestamp]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[singletsinsertorc]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_literal]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_to_utc_timestamp]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[update_all_types]
(batchId=18)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_timestamp]
(batchId=103)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_2]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part_all_primitive_llap_io]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part_all_primitive_llap_io]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_date]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_timestamp_funcs]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge5]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge6]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge7]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge8]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge9]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_incompat1]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[acid_vectorization_original_tez]
(batchId=107)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[orc_merge12]
(batchId=106)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_timestamp_funcs]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query12]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query20]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query21]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query32]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query37]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query40]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query5]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query77]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query80]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query82]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query92]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query95]
(batchId=257)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query98]
(batchId=257)
[jira] [Updated] (HIVE-19646) Filesystem closed error in HiveProtoLoggingHook
[
https://issues.apache.org/jira/browse/HIVE-19646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Jaiprakash updated HIVE-19646:
-
Attachment: HIVE-19646.07-branch-3.patch
> Filesystem closed error in HiveProtoLoggingHook
> ---
>
> Key: HIVE-19646
> URL: https://issues.apache.org/jira/browse/HIVE-19646
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
>Priority: Major
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19646.01-branch-3.patch, HIVE-19646.01.patch,
> HIVE-19646.02.patch, HIVE-19646.03-branch-3.patch, HIVE-19646.03.patch,
> HIVE-19646.04.patch, HIVE-19646.05-branch-3.patch, HIVE-19646.05.patch,
> HIVE-19646.06-branch-3.patch, HIVE-19646.07-branch-3.patch
>
>
> Exception in proto logging hook on secure cluster.
> {code}
> 2018-05-18T04:48:01,136 ERROR [Hive Hook Proto Log Writer 0]:
> hooks.HiveProtoLoggingHook (:()) - Error writing proto message for query
> hive_20180518043717_ca3ab4df-6cab-4920-aa44-2340ae246ad2, eventType:
> QUERY_SUBMITTED:
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:472)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1652)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734)
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getPathForDate(DatePartitionedLogger.java:89)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getWriter(DatePartitionedLogger.java:73)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.writeEvent(HiveProtoLoggingHook.java:283)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.lambda$generateEvent$1(HiveProtoLoggingHook.java:274)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19462) Fix mapping for char_length function to enable pushdown to Druid.
[
https://issues.apache.org/jira/browse/HIVE-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499052#comment-16499052
]
Hive QA commented on HIVE-19462:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925805/HIVE-19462.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14443 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_aggregate]
(batchId=159)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11437/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11437/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11437/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925805 - PreCommit-HIVE-Build
> Fix mapping for char_length function to enable pushdown to Druid.
> --
>
> Key: HIVE-19462
> URL: https://issues.apache.org/jira/browse/HIVE-19462
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19462.2.patch, HIVE-19462.3.patch,
> HIVE-19462.4.patch, HIVE-19462.patch
>
>
> currently char_length is not push down to Druid because of missing mapping
> form/to calcite
> This patch will add this mapping.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19462) Fix mapping for char_length function to enable pushdown to Druid.
[
https://issues.apache.org/jira/browse/HIVE-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499014#comment-16499014
]
Hive QA commented on HIVE-19462:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11437/dev-support/hive-personality.sh
|
| git revision | master / 4463c2b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11437/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix mapping for char_length function to enable pushdown to Druid.
> --
>
> Key: HIVE-19462
> URL: https://issues.apache.org/jira/browse/HIVE-19462
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19462.2.patch, HIVE-19462.3.patch,
> HIVE-19462.4.patch, HIVE-19462.patch
>
>
> currently char_length is not push down to Druid because of missing mapping
> form/to calcite
> This patch will add this mapping.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19237) Only use an operatorId once in a plan
[
https://issues.apache.org/jira/browse/HIVE-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499008#comment-16499008
]
Hive QA commented on HIVE-19237:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925901/HIVE-19237.09.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11436/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11436/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11436/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925901/HIVE-19237.09.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925901 - PreCommit-HIVE-Build
> Only use an operatorId once in a plan
> -
>
> Key: HIVE-19237
> URL: https://issues.apache.org/jira/browse/HIVE-19237
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19237.01.patch, HIVE-19237.02.patch,
> HIVE-19237.03.patch, HIVE-19237.04.patch, HIVE-19237.05.patch,
> HIVE-19237.05.patch, HIVE-19237.06.patch, HIVE-19237.07.patch,
> HIVE-19237.08.patch, HIVE-19237.08.patch, HIVE-19237.09.patch
>
>
> Column stats autogather plan part is added from a plan compiled by the driver
> itself; however that driver starts to use operatorIds from 1 ; so it's
> possible that 2 SEL_1 operators end up in the same plan...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18684) Race condition in RemoteSparkJobMonitor
[
https://issues.apache.org/jira/browse/HIVE-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499007#comment-16499007
]
Hive QA commented on HIVE-18684:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925801/HIVE-18684.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 14445 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers1
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerShortQueryElapsedTime
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=242)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11435/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11435/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11435/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925801 - PreCommit-HIVE-Build
> Race condition in RemoteSparkJobMonitor
> ---
>
> Key: HIVE-18684
> URL: https://issues.apache.org/jira/browse/HIVE-18684
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18684.1.patch, HIVE-18684.2.patch
>
>
> There is a race condition in {{RemoteSparkJobMonitor}}. Sometimes the info in
> {{RemoteSparkJobMonitor#startMonitor.STARTED}} gets printed out, sometimes it
> doesn't. This can be easily verified by running a qtest on
> {{TestMiniSparkOnYarnCliDriver}} and counting the number of times {{Query
> Hive on Spark job}} is printed vs. the number of times {{Finished
> successfully in}} gets printed.
> The issue is that {{RemoteSparkJobMonitor}} runs every one second, and checks
> the state of {{JobHandle}}. Depending on the state, it prints out some
> logging info. The content of the logs contain an implicit assumption that
> logs in the {{STARTED}} state are printed before the logs in the
> {{SUCCEEDED}} state. However, this isn't always the case. The state
> transitions are driven by how long the remote Spark job takes to run, and it
> it finishes within one second then the logs in the {{STARTED}} state never
> printed.
> This can be confusing to users, and there is key debugging information that
> is printed in the {{STARTED}} state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19602) Refactor inplace progress code in Hive-on-spark progress monitor to use ProgressMonitor instance
[ https://issues.apache.org/jira/browse/HIVE-19602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498996#comment-16498996 ] Rui Li commented on HIVE-19602: --- Thanks for pinging me. +1 pending test > Refactor inplace progress code in Hive-on-spark progress monitor to use > ProgressMonitor instance > > > Key: HIVE-19602 > URL: https://issues.apache.org/jira/browse/HIVE-19602 > Project: Hive > Issue Type: Bug >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19602.3.patch > > > We can refactor the HOS inplace progress monitor code > (SparkJobMonitor#printStatusInPlace) to use InplaceUpdate#render. > We can create an instance of ProgressMonitor and use it to show the progress. > This would be similar to : > [https://github.com/apache/hive/blob/0b6bea89f74b607299ad944b37e4b62c711aaa69/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java#L181] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18684) Race condition in RemoteSparkJobMonitor
[
https://issues.apache.org/jira/browse/HIVE-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498976#comment-16498976
]
Hive QA commented on HIVE-18684:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
28s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} ql: The patch generated 16 new + 2 unchanged - 4 fixed
= 18 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
11s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11435/dev-support/hive-personality.sh
|
| git revision | master / 4463c2b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11435/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11435/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11435/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Race condition in RemoteSparkJobMonitor
> ---
>
> Key: HIVE-18684
> URL: https://issues.apache.org/jira/browse/HIVE-18684
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18684.1.patch, HIVE-18684.2.patch
>
>
> There is a race condition in {{RemoteSparkJobMonitor}}. Sometimes the info in
> {{RemoteSparkJobMonitor#startMonitor.STARTED}} gets printed out, sometimes it
> doesn't. This can be easily verified by running a qtest on
> {{TestMiniSparkOnYarnCliDriver}} and counting the number of times {{Query
> Hive on Spark job}} is printed vs. the number of times {{Finished
> successfully in}} gets printed.
> The issue is that {{RemoteSparkJobMonitor}} runs every one second, and checks
> the state of {{JobHandle}}. Depending on the state, it prints out some
> logging info. The content of the logs contain an implicit assumption that
> logs in the {{STARTED}} state are printed before the logs in the
> {{SUCCEEDED}} state. However, this isn't always the case. The state
> transitions are driven by how long the remote Spark job takes to run, and it
> it finishes within one second then the logs in the {{STARTED}} state never
> printed.
> This can be confusing to users, and there is key debugging information that
> is
[jira] [Commented] (HIVE-19605) TAB_COL_STATS table has no index on db/table name
[
https://issues.apache.org/jira/browse/HIVE-19605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498963#comment-16498963
]
Hive QA commented on HIVE-19605:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925786/HIVE-19605.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14443 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestRuntimeStats.testCleanup[Embedded]
(batchId=211)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11434/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11434/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11434/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925786 - PreCommit-HIVE-Build
> TAB_COL_STATS table has no index on db/table name
> -
>
> Key: HIVE-19605
> URL: https://issues.apache.org/jira/browse/HIVE-19605
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Todd Lipcon
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19605.01.patch
>
>
> The TAB_COL_STATS table is missing an index on (CAT_NAME, DB_NAME,
> TABLE_NAME). The getTableColumnStatistics call queries based on this tuple.
> This makes those queries take a significant amount of time in large
> metastores since they do a full table scan.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19605) TAB_COL_STATS table has no index on db/table name
[
https://issues.apache.org/jira/browse/HIVE-19605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498946#comment-16498946
]
Hive QA commented on HIVE-19605:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11434/dev-support/hive-personality.sh
|
| git revision | master / 4463c2b |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11434/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TAB_COL_STATS table has no index on db/table name
> -
>
> Key: HIVE-19605
> URL: https://issues.apache.org/jira/browse/HIVE-19605
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Todd Lipcon
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19605.01.patch
>
>
> The TAB_COL_STATS table is missing an index on (CAT_NAME, DB_NAME,
> TABLE_NAME). The getTableColumnStatistics call queries based on this tuple.
> This makes those queries take a significant amount of time in large
> metastores since they do a full table scan.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19747) "GRANT ALL TO USER" failed with NullPointerException
[
https://issues.apache.org/jira/browse/HIVE-19747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damien Carol updated HIVE-19747:
Description:
If you issue the command
{code:sql}
grant all to user abc
{code}
you will see the following NPE exception. Seems the type in hivePrivObject is
not initialized.
{noformat}
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.isOwner(SQLAuthorizationUtils.java:265)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromMetaStore(SQLAuthorizationUtils.java:212)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.GrantPrivAuthUtils.checkRequiredPrivileges(GrantPrivAuthUtils.java:64)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.GrantPrivAuthUtils.authorize(GrantPrivAuthUtils.java:50)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAccessController.grantPrivileges(SQLStdHiveAccessController.java:179)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAccessControllerWrapper.grantPrivileges(SQLStdHiveAccessControllerWrapper.java:70)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl.grantPrivileges(HiveAuthorizerImpl.java:48)
at
org.apache.hadoop.hive.ql.exec.DDLTask.grantOrRevokePrivileges(DDLTask.java:1123
{noformat}
was:
If you issue the command 'grant all to user abc', you will see the following
NPE exception. Seems the type in hivePrivObject is not initialized.
{noformat}
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.isOwner(SQLAuthorizationUtils.java:265)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromMetaStore(SQLAuthorizationUtils.java:212)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.GrantPrivAuthUtils.checkRequiredPrivileges(GrantPrivAuthUtils.java:64)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.GrantPrivAuthUtils.authorize(GrantPrivAuthUtils.java:50)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAccessController.grantPrivileges(SQLStdHiveAccessController.java:179)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAccessControllerWrapper.grantPrivileges(SQLStdHiveAccessControllerWrapper.java:70)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl.grantPrivileges(HiveAuthorizerImpl.java:48)
at
org.apache.hadoop.hive.ql.exec.DDLTask.grantOrRevokePrivileges(DDLTask.java:1123
{noformat}
> "GRANT ALL TO USER" failed with NullPointerException
>
>
> Key: HIVE-19747
> URL: https://issues.apache.org/jira/browse/HIVE-19747
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Affects Versions: 2.1.0
>Reporter: Aihua Xu
>Priority: Minor
>
> If you issue the command
> {code:sql}
> grant all to user abc
> {code}
> you will see the following NPE exception. Seems the type in hivePrivObject is
> not initialized.
> {noformat}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.isOwner(SQLAuthorizationUtils.java:265)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromMetaStore(SQLAuthorizationUtils.java:212)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.GrantPrivAuthUtils.checkRequiredPrivileges(GrantPrivAuthUtils.java:64)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.GrantPrivAuthUtils.authorize(GrantPrivAuthUtils.java:50)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAccessController.grantPrivileges(SQLStdHiveAccessController.java:179)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAccessControllerWrapper.grantPrivileges(SQLStdHiveAccessControllerWrapper.java:70)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl.grantPrivileges(HiveAuthorizerImpl.java:48)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.grantOrRevokePrivileges(DDLTask.java:1123
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19237) Only use an operatorId once in a plan
[
https://issues.apache.org/jira/browse/HIVE-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498944#comment-16498944
]
Hive QA commented on HIVE-19237:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925901/HIVE-19237.09.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11433/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11433/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11433/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925901/HIVE-19237.09.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925901 - PreCommit-HIVE-Build
> Only use an operatorId once in a plan
> -
>
> Key: HIVE-19237
> URL: https://issues.apache.org/jira/browse/HIVE-19237
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19237.01.patch, HIVE-19237.02.patch,
> HIVE-19237.03.patch, HIVE-19237.04.patch, HIVE-19237.05.patch,
> HIVE-19237.05.patch, HIVE-19237.06.patch, HIVE-19237.07.patch,
> HIVE-19237.08.patch, HIVE-19237.08.patch, HIVE-19237.09.patch
>
>
> Column stats autogather plan part is added from a plan compiled by the driver
> itself; however that driver starts to use operatorIds from 1 ; so it's
> possible that 2 SEL_1 operators end up in the same plan...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19730) fix TestTablesGetExists's flakiness
[
https://issues.apache.org/jira/browse/HIVE-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498943#comment-16498943
]
Hive QA commented on HIVE-19730:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925957/HIVE-19730.2.patch
{color:green}SUCCESS:{color} +1 due to 22 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14443 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema[5]
(batchId=198)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11432/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11432/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11432/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925957 - PreCommit-HIVE-Build
> fix TestTablesGetExists's flakiness
> ---
>
> Key: HIVE-19730
> URL: https://issues.apache.org/jira/browse/HIVE-19730
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19730.2.patch, HIVE-19730.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18990) Hive doesn't close Tez session properly
[ https://issues.apache.org/jira/browse/HIVE-18990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498939#comment-16498939 ] Igor Kryvenko commented on HIVE-18990: -- [~jdere] Hi. Can you provide the query to reproduce it? I'll investigate it > Hive doesn't close Tez session properly > --- > > Key: HIVE-18990 > URL: https://issues.apache.org/jira/browse/HIVE-18990 > Project: Hive > Issue Type: Bug >Reporter: Igor Kryvenko >Assignee: Igor Kryvenko >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18990.01.patch, HIVE-18990.02.patch, > HIVE-18990.03.patch > > > Hive doesn't close Tez session properly if AM isn't ready for accepting DAG. > *STR* > This can be easily reproduced using the following steps: > *1) configure cluster on Tez;* > *2) create file test.hql* > cat ~/test.hql > show databases; > *3) run the job* > $ hive --hiveconf hive.root.logger=DEBUG,console --hiveconf > hive.execution.engine=tez -f ~/test.hql > If we login into Yarn UI, we will see that jobs status is failed even it > finished successfully. > It happens because hive creates tez session by default. And if query finished > very quickly, we can't close tez session properly because AM isn't ready for > accepting any requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19730) fix TestTablesGetExists's flakiness
[
https://issues.apache.org/jira/browse/HIVE-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498930#comment-16498930
]
Hive QA commented on HIVE-19730:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
44s{color} | {color:blue} standalone-metastore in master has 214 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11432/dev-support/hive-personality.sh
|
| git revision | master / 4463c2b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11432/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> fix TestTablesGetExists's flakiness
> ---
>
> Key: HIVE-19730
> URL: https://issues.apache.org/jira/browse/HIVE-19730
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19730.2.patch, HIVE-19730.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19584) Dictionary encoding for string types
[
https://issues.apache.org/jira/browse/HIVE-19584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498923#comment-16498923
]
Hive QA commented on HIVE-19584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12926023/HIVE-19584.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 14445 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=242)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11431/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11431/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11431/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12926023 - PreCommit-HIVE-Build
> Dictionary encoding for string types
>
>
> Key: HIVE-19584
> URL: https://issues.apache.org/jira/browse/HIVE-19584
> Project: Hive
> Issue Type: Sub-task
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19584.1.patch, HIVE-19584.2.patch,
> HIVE-19584.3.patch, HIVE-19584.4.patch
>
>
> Apache Arrow supports dictionary encoding for some data types. So implement
> dictionary encoding for string types in Arrow SerDe.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19584) Dictionary encoding for string types
[
https://issues.apache.org/jira/browse/HIVE-19584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498916#comment-16498916
]
Hive QA commented on HIVE-19584:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
31s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 149 new + 551 unchanged - 50
fixed = 700 total (was 601) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
40s{color} | {color:red} ql generated 2 new + 2278 unchanged - 0 fixed = 2280
total (was 2278) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Unchecked/unconfirmed cast from org.apache.arrow.vector.FieldVector to
org.apache.arrow.vector.complex.MapVector in
org.apache.hadoop.hive.ql.io.arrow.Serializer.write(FieldVector, ColumnVector,
TypeInfo, int, boolean) At
Serializer.java:org.apache.arrow.vector.complex.MapVector in
org.apache.hadoop.hive.ql.io.arrow.Serializer.write(FieldVector, ColumnVector,
TypeInfo, int, boolean) At Serializer.java:[line 281] |
| | Unchecked/unconfirmed cast from org.apache.arrow.vector.complex.MapVector
to org.apache.arrow.vector.FieldVector in
org.apache.hadoop.hive.ql.io.arrow.Serializer.writeStruct(MapVector,
StructColumnVector, StructTypeInfo, int, boolean) At
Serializer.java:org.apache.arrow.vector.FieldVector in
org.apache.hadoop.hive.ql.io.arrow.Serializer.writeStruct(MapVector,
StructColumnVector, StructTypeInfo, int, boolean) At Serializer.java:[line
356] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11431/dev-support/hive-personality.sh
|
| git revision | master / 4463c2b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11431/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11431/yetus/new-findbugs-ql.html
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11431/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Dictionary
[jira] [Updated] (HIVE-19529) Vectorization: Date/Timestamp NULL issues
[ https://issues.apache.org/jira/browse/HIVE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19529: Fix Version/s: 3.1.0 > Vectorization: Date/Timestamp NULL issues > - > > Key: HIVE-19529 > URL: https://issues.apache.org/jira/browse/HIVE-19529 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19529.06-branch-3.patch, HIVE-19529.06.patch > > > Wrong results found for: > date_add/date_sub > UT areas: > date_add/date_sub > datediff > to_date > interval_year_month + interval_year_month > interval_day_time + interval_day_time > interval_day_time + timestamp > timestamp + interval_day_time > date + interval_day_time > interval_day_time + date > interval_year_month + date > date + interval_year_month > interval_year_month + interval_year_month > timestamp + interval_year_month > date - date > interval_year_month - interval_year_month > interval_day_time - interval_day_time > timestamp - interval_day_time > timestamp - timestamp > date - timestamp > timestamp - date > date - interval_day_time > date - interval_year_month > timestamp - interval_year_month -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19529) Vectorization: Date/Timestamp NULL issues
[ https://issues.apache.org/jira/browse/HIVE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498910#comment-16498910 ] Matt McCline commented on HIVE-19529: - Committed to branch-3 > Vectorization: Date/Timestamp NULL issues > - > > Key: HIVE-19529 > URL: https://issues.apache.org/jira/browse/HIVE-19529 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19529.06-branch-3.patch, HIVE-19529.06.patch > > > Wrong results found for: > date_add/date_sub > UT areas: > date_add/date_sub > datediff > to_date > interval_year_month + interval_year_month > interval_day_time + interval_day_time > interval_day_time + timestamp > timestamp + interval_day_time > date + interval_day_time > interval_day_time + date > interval_year_month + date > date + interval_year_month > interval_year_month + interval_year_month > timestamp + interval_year_month > date - date > interval_year_month - interval_year_month > interval_day_time - interval_day_time > timestamp - interval_day_time > timestamp - timestamp > date - timestamp > timestamp - date > date - interval_day_time > date - interval_year_month > timestamp - interval_year_month -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19529) Vectorization: Date/Timestamp NULL issues
[ https://issues.apache.org/jira/browse/HIVE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498909#comment-16498909 ] Matt McCline commented on HIVE-19529: - Test failures are unrelated. > Vectorization: Date/Timestamp NULL issues > - > > Key: HIVE-19529 > URL: https://issues.apache.org/jira/browse/HIVE-19529 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 4.0.0 > > Attachments: HIVE-19529.06-branch-3.patch, HIVE-19529.06.patch > > > Wrong results found for: > date_add/date_sub > UT areas: > date_add/date_sub > datediff > to_date > interval_year_month + interval_year_month > interval_day_time + interval_day_time > interval_day_time + timestamp > timestamp + interval_day_time > date + interval_day_time > interval_day_time + date > interval_year_month + date > date + interval_year_month > interval_year_month + interval_year_month > timestamp + interval_year_month > date - date > interval_year_month - interval_year_month > interval_day_time - interval_day_time > timestamp - interval_day_time > timestamp - timestamp > date - timestamp > timestamp - date > date - interval_day_time > date - interval_year_month > timestamp - interval_year_month -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19646) Filesystem closed error in HiveProtoLoggingHook
[
https://issues.apache.org/jira/browse/HIVE-19646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498907#comment-16498907
]
Hive QA commented on HIVE-19646:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925698/HIVE-19646.06-branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11430/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11430/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11430/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12925698/HIVE-19646.06-branch-3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925698 - PreCommit-HIVE-Build
> Filesystem closed error in HiveProtoLoggingHook
> ---
>
> Key: HIVE-19646
> URL: https://issues.apache.org/jira/browse/HIVE-19646
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
>Priority: Major
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19646.01-branch-3.patch, HIVE-19646.01.patch,
> HIVE-19646.02.patch, HIVE-19646.03-branch-3.patch, HIVE-19646.03.patch,
> HIVE-19646.04.patch, HIVE-19646.05-branch-3.patch, HIVE-19646.05.patch,
> HIVE-19646.06-branch-3.patch
>
>
> Exception in proto logging hook on secure cluster.
> {code}
> 2018-05-18T04:48:01,136 ERROR [Hive Hook Proto Log Writer 0]:
> hooks.HiveProtoLoggingHook (:()) - Error writing proto message for query
> hive_20180518043717_ca3ab4df-6cab-4920-aa44-2340ae246ad2, eventType:
> QUERY_SUBMITTED:
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:472)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1652)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734)
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
> at
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getPathForDate(DatePartitionedLogger.java:89)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getWriter(DatePartitionedLogger.java:73)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.writeEvent(HiveProtoLoggingHook.java:283)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.lambda$generateEvent$1(HiveProtoLoggingHook.java:274)
> ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19529) Vectorization: Date/Timestamp NULL issues
[
https://issues.apache.org/jira/browse/HIVE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498906#comment-16498906
]
Hive QA commented on HIVE-19529:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12925937/HIVE-19529.06-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14379 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=253)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=305)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11429/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11429/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11429/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12925937 - PreCommit-HIVE-Build
> Vectorization: Date/Timestamp NULL issues
> -
>
> Key: HIVE-19529
> URL: https://issues.apache.org/jira/browse/HIVE-19529
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Blocker
> Fix For: 4.0.0
>
> Attachments: HIVE-19529.06-branch-3.patch, HIVE-19529.06.patch
>
>
> Wrong results found for:
> date_add/date_sub
> UT areas:
> date_add/date_sub
> datediff
> to_date
> interval_year_month + interval_year_month
> interval_day_time + interval_day_time
> interval_day_time + timestamp
> timestamp + interval_day_time
> date + interval_day_time
> interval_day_time + date
> interval_year_month + date
> date + interval_year_month
> interval_year_month + interval_year_month
> timestamp + interval_year_month
> date - date
> interval_year_month - interval_year_month
> interval_day_time - interval_day_time
> timestamp - interval_day_time
> timestamp - timestamp
> date - timestamp
> timestamp - date
> date - interval_day_time
> date - interval_year_month
> timestamp - interval_year_month
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19668) Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and duplicate strings
[
https://issues.apache.org/jira/browse/HIVE-19668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misha Dmitriev updated HIVE-19668:
--
Attachment: HIVE-19668.01.patch
> Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and
> duplicate strings
> --
>
> Key: HIVE-19668
> URL: https://issues.apache.org/jira/browse/HIVE-19668
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Misha Dmitriev
>Assignee: Misha Dmitriev
>Priority: Major
> Attachments: HIVE-19668.01.patch, image-2018-05-22-17-41-39-572.png
>
>
> I've recently analyzed a HS2 heap dump, obtained when there was a huge memory
> spike during compilation of some big query. The analysis was done with jxray
> ([www.jxray.com).|http://www.jxray.com)./] It turns out that more than 90% of
> the 20G heap was used by data structures associated with query parsing
> ({{org.apache.hadoop.hive.ql.parse.QBExpr}}). There are probably multiple
> opportunities for optimizations here. One of them is to stop the code from
> creating duplicate instances of {{org.antlr.runtime.CommonToken}} class. See
> a sample of these objects in the attached image:
> !image-2018-05-22-17-41-39-572.png|width=879,height=399!
> Looks like these particular {{CommonToken}} objects are constants, that don't
> change once created. I see some code, e.g. in
> {{org.apache.hadoop.hive.ql.parse.CalcitePlanner}}, where such objects are
> apparently repeatedly created with e.g. {{new
> CommonToken(HiveParser.TOK_INSERT, "TOK_INSERT")}} If these 33 token kinds
> are instead created once and reused, we will save more than 1/10th of the
> heap in this scenario. Plus, since these objects are small but very numerous,
> getting rid of them will remove a gread deal of pressure from the GC.
> Another source of waste are duplicate strings, that collectively waste 26.1%
> of memory. Some of them come from CommonToken objects that have the same text
> (i.e. for multiple CommonToken objects the contents of their 'text' Strings
> are the same, but each has its own copy of that String). Other duplicate
> strings come from other sources, that are easy enough to fix by adding
> String.intern() calls.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19668) Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and duplicate strings
[
https://issues.apache.org/jira/browse/HIVE-19668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misha Dmitriev updated HIVE-19668:
--
Status: Patch Available (was: In Progress)
> Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and
> duplicate strings
> --
>
> Key: HIVE-19668
> URL: https://issues.apache.org/jira/browse/HIVE-19668
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Misha Dmitriev
>Assignee: Misha Dmitriev
>Priority: Major
> Attachments: HIVE-19668.01.patch, image-2018-05-22-17-41-39-572.png
>
>
> I've recently analyzed a HS2 heap dump, obtained when there was a huge memory
> spike during compilation of some big query. The analysis was done with jxray
> ([www.jxray.com).|http://www.jxray.com)./] It turns out that more than 90% of
> the 20G heap was used by data structures associated with query parsing
> ({{org.apache.hadoop.hive.ql.parse.QBExpr}}). There are probably multiple
> opportunities for optimizations here. One of them is to stop the code from
> creating duplicate instances of {{org.antlr.runtime.CommonToken}} class. See
> a sample of these objects in the attached image:
> !image-2018-05-22-17-41-39-572.png|width=879,height=399!
> Looks like these particular {{CommonToken}} objects are constants, that don't
> change once created. I see some code, e.g. in
> {{org.apache.hadoop.hive.ql.parse.CalcitePlanner}}, where such objects are
> apparently repeatedly created with e.g. {{new
> CommonToken(HiveParser.TOK_INSERT, "TOK_INSERT")}} If these 33 token kinds
> are instead created once and reused, we will save more than 1/10th of the
> heap in this scenario. Plus, since these objects are small but very numerous,
> getting rid of them will remove a gread deal of pressure from the GC.
> Another source of waste are duplicate strings, that collectively waste 26.1%
> of memory. Some of them come from CommonToken objects that have the same text
> (i.e. for multiple CommonToken objects the contents of their 'text' Strings
> are the same, but each has its own copy of that String). Other duplicate
> strings come from other sources, that are easy enough to fix by adding
> String.intern() calls.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-19668) Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and duplicate strings
[
https://issues.apache.org/jira/browse/HIVE-19668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-19668 started by Misha Dmitriev.
-
> Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and
> duplicate strings
> --
>
> Key: HIVE-19668
> URL: https://issues.apache.org/jira/browse/HIVE-19668
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Misha Dmitriev
>Assignee: Misha Dmitriev
>Priority: Major
> Attachments: image-2018-05-22-17-41-39-572.png
>
>
> I've recently analyzed a HS2 heap dump, obtained when there was a huge memory
> spike during compilation of some big query. The analysis was done with jxray
> ([www.jxray.com).|http://www.jxray.com)./] It turns out that more than 90% of
> the 20G heap was used by data structures associated with query parsing
> ({{org.apache.hadoop.hive.ql.parse.QBExpr}}). There are probably multiple
> opportunities for optimizations here. One of them is to stop the code from
> creating duplicate instances of {{org.antlr.runtime.CommonToken}} class. See
> a sample of these objects in the attached image:
> !image-2018-05-22-17-41-39-572.png|width=879,height=399!
> Looks like these particular {{CommonToken}} objects are constants, that don't
> change once created. I see some code, e.g. in
> {{org.apache.hadoop.hive.ql.parse.CalcitePlanner}}, where such objects are
> apparently repeatedly created with e.g. {{new
> CommonToken(HiveParser.TOK_INSERT, "TOK_INSERT")}} If these 33 token kinds
> are instead created once and reused, we will save more than 1/10th of the
> heap in this scenario. Plus, since these objects are small but very numerous,
> getting rid of them will remove a gread deal of pressure from the GC.
> Another source of waste are duplicate strings, that collectively waste 26.1%
> of memory. Some of them come from CommonToken objects that have the same text
> (i.e. for multiple CommonToken objects the contents of their 'text' Strings
> are the same, but each has its own copy of that String). Other duplicate
> strings come from other sources, that are easy enough to fix by adding
> String.intern() calls.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HIVE-19529) Vectorization: Date/Timestamp NULL issues
[ https://issues.apache.org/jira/browse/HIVE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19529: Comment: was deleted (was: #11429 ?) > Vectorization: Date/Timestamp NULL issues > - > > Key: HIVE-19529 > URL: https://issues.apache.org/jira/browse/HIVE-19529 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 4.0.0 > > Attachments: HIVE-19529.06-branch-3.patch, HIVE-19529.06.patch > > > Wrong results found for: > date_add/date_sub > UT areas: > date_add/date_sub > datediff > to_date > interval_year_month + interval_year_month > interval_day_time + interval_day_time > interval_day_time + timestamp > timestamp + interval_day_time > date + interval_day_time > interval_day_time + date > interval_year_month + date > date + interval_year_month > interval_year_month + interval_year_month > timestamp + interval_year_month > date - date > interval_year_month - interval_year_month > interval_day_time - interval_day_time > timestamp - interval_day_time > timestamp - timestamp > date - timestamp > timestamp - date > date - interval_day_time > date - interval_year_month > timestamp - interval_year_month -- This message was sent by Atlassian JIRA (v7.6.3#76005)
