[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348135#comment-16348135 ] Hive QA commented on HIVE-18552: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12908625/HIVE-18552.4.patch {color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 12966 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=240) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=167) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=171) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes] (batchId=163) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat] (batchId=180) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] (batchId=122) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=221) org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded] (batchId=206) org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded] (batchId=206) org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap (batchId=282) org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256) org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8961/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8961/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8961/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 21 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12908625 - PreCommit-HIVE-Build > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch, HIVE-18552.2.patch, > HIVE-18552.3.patch, HIVE-18552.4.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348087#comment-16348087 ] Hive QA commented on HIVE-18552: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 31s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 16m 9s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / 419593e | | Default Java | 1.8.0_111 | | modules | C: common ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8961/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch, HIVE-18552.2.patch, > HIVE-18552.3.patch, HIVE-18552.4.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347653#comment-16347653 ] Vihang Karajgaonkar commented on HIVE-18552: patch looks good to me. +1 Since this removes an existing config, we should only merge it in Hive 3.0.0 > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch, HIVE-18552.2.patch, > HIVE-18552.3.patch, HIVE-18552.4.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347327#comment-16347327 ] Sahil Takiar commented on HIVE-18552: - [~aihuaxu], [~vihangk1] could you take a look? > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch, HIVE-18552.2.patch, > HIVE-18552.3.patch, HIVE-18552.4.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346351#comment-16346351 ] Hive QA commented on HIVE-18552: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12908453/HIVE-18552.3.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 12863 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=240) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=49) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook] (batchId=13) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=152) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=167) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=171) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin] (batchId=166) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes] (batchId=163) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] (batchId=122) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=221) org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionWithAuthInfoNoDbName[Embedded] (batchId=206) org.apache.hadoop.hive.metastore.client.TestTablesGetExists.testGetAllTablesCaseInsensitive[Embedded] (batchId=206) org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded] (batchId=206) org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap (batchId=282) org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256) org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testCheckAggOrWindowing (batchId=276) org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testExtractConjuncts (batchId=276) org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testExtractSubQueries (batchId=276) org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testRewriteOuterQueryWhere (batchId=276) org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testRewriteOuterQueryWhere2 (batchId=276) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testRenamePartitionWithCM (batchId=228) org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8940/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8940/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8940/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 31 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12908453 - PreCommit-HIVE-Build > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch, HIVE-18552.2.patch, > HIVE-18552.3.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore,
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346317#comment-16346317 ] Hive QA commented on HIVE-18552: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 30s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 15m 46s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / 3e4adaa | | Default Java | 1.8.0_111 | | modules | C: common ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8940/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch, HIVE-18552.2.patch, > HIVE-18552.3.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344510#comment-16344510 ] Hive QA commented on HIVE-18552: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12908249/HIVE-18552.1.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 12795 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=240) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=152) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=167) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=171) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_bmj_schema_evolution] (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes] (batchId=163) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_view_failure6] (batchId=95) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_view_failure6_2] (batchId=95) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_double_bigint] (batchId=94) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_double_bigint_2] (batchId=94) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_string_bigint] (batchId=95) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_string_bigint_2] (batchId=95) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input4] (batchId=95) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input4_2] (batchId=95) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input_part0_neg] (batchId=94) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input_part0_neg_2] (batchId=95) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] (batchId=122) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=221) org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap (batchId=282) org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256) org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8918/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8918/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8918/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 30 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12908249 - PreCommit-HIVE-Build > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344475#comment-16344475 ] Hive QA commented on HIVE-18552: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 37s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 45s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 50s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 16m 19s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / be8a451 | | Default Java | 1.8.0_111 | | modules | C: common ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8918/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Split hive.strict.checks.large.query into two configs > - > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18552.1.patch > > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs, enable partition filter check by default
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344366#comment-16344366 ] Sahil Takiar commented on HIVE-18552: - [~sershe] sounds reasonable. Didn't realize there was an existing config to control the max # of partitions to scan. Will post a patch that splits {{hive.strict.checks.large.query}} and keeps both the new ones false by default. > Split hive.strict.checks.large.query into two configs, enable partition > filter check by default > --- > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs, enable partition filter check by default
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340441#comment-16340441 ] Sergey Shelukhin commented on HIVE-18552: - It makes sense to separate them, but requiring a partitioned filter should not be on by default because it fails many legitimate queries. IIRC there's a separate setting that controls the max # of partitions to allow in a scan > Split hive.strict.checks.large.query into two configs, enable partition > filter check by default > --- > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs, enable partition filter check by default
[ https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340428#comment-16340428 ] Sahil Takiar commented on HIVE-18552: - [~sershe], [~ashutoshc] any thoughts on this? > Split hive.strict.checks.large.query into two configs, enable partition > filter check by default > --- > > Key: HIVE-18552 > URL: https://issues.apache.org/jira/browse/HIVE-18552 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > > {{hive.strict.checks.large.query}} controls the strict checks for restricting > order bys with no limits, and scans of a partitioned table without a filter > on the partition table. > While both checks prevent "large" queries from being run, they both control > very different behavior. It would be better if users could control these > restrictions separately. > Furthermore, many users make the mistake of abusing partitioned tables and > often end up in a situation where they are running queries that are doing > full-table scans of partitioned tables. This can lead to lots of issues for > Hive - e.g. OOM issues because so many partitions are loaded in memory. So it > would be good if we enabled this restriction by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)