[jira] [Updated] (HIVE-13563) Hive Streaming does not honor orc.compress.size and orc.stripe.size table properties
[ https://issues.apache.org/jira/browse/HIVE-13563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-13563: -- Labels: (was: TODOC2.1) > Hive Streaming does not honor orc.compress.size and orc.stripe.size table > properties > > > Key: HIVE-13563 > URL: https://issues.apache.org/jira/browse/HIVE-13563 > Project: Hive > Issue Type: Bug > Components: ORC >Affects Versions: 2.1.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Fix For: 1.3.0, 2.1.0 > > Attachments: HIVE-13563.1.patch, HIVE-13563.2.patch, > HIVE-13563.3.patch, HIVE-13563.4.patch, HIVE-13563.branch-1.patch > > > According to the doc: > https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC#LanguageManualORC-HiveQLSyntax > One should be able to specify tblproperties for many ORC options. > But the settings for orc.compress.size and orc.stripe.size don't take effect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8960) ParsingException in the WHERE statement with a Sub Query
[ https://issues.apache.org/jira/browse/HIVE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429608#comment-15429608 ] Lefty Leverenz commented on HIVE-8960: -- [~Vikash Pareek], perhaps you would get more replies if you posted this question on the u...@hive.apache.org mailing list. * [Hive mailing lists | http://hive.apache.org/mailing_lists.html] > ParsingException in the WHERE statement with a Sub Query > > > Key: HIVE-8960 > URL: https://issues.apache.org/jira/browse/HIVE-8960 > Project: Hive > Issue Type: Bug > Components: Parser >Affects Versions: 0.13.0 > Environment: Secured HDP 2.1.3 with Hive 0.13.0 >Reporter: Rémy SAISSY > > Comparison with a Sub query in a WHERE statement does not work. > Given that id_chargement is an integer: > USE db1; > SELECT * FROM tbl1 a WHERE a.id_chargement > (SELECT MAX(b.id_chargement) > FROM tbl2 b); > or > SELECT * FROM tbl1 a WHERE a.id_chargement > (SELECT b.id_chargement FROM > tbl2 b LIMIT 1); > Both return the following parsing error: > Error: Error while compiling statement: FAILED: ParseException line 1:88 > cannot recognize input near 'SELECT' 'b' '.' in expression specification > (state=42000,code=4) > java.sql.SQLException: Error while compiling statement: FAILED: > ParseException line 1:88 cannot recognize input near 'SELECT' 'b' '.' in > expression specification > at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121) > at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109) > at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231) > at org.apache.hive.beeline.Commands.execute(Commands.java:736) > at org.apache.hive.beeline.Commands.sql(Commands.java:657) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:804) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:659) > at > org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:368) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:351) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14272) ConditionalResolverMergeFiles should keep staging data on HDFS, then copy (no rename) to S3
[
https://issues.apache.org/jira/browse/HIVE-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abdullah Yousufi updated HIVE-14272:
Assignee: Sergio Peña (was: Abdullah Yousufi)
> ConditionalResolverMergeFiles should keep staging data on HDFS, then copy (no
> rename) to S3
> ---
>
> Key: HIVE-14272
> URL: https://issues.apache.org/jira/browse/HIVE-14272
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>
> If {{hive.merge.mapfiles}} is True, and the output table to write is on S3,
> then Hive will generate a conditional plan where smaller files will be merged
> into larger sizes.
> If the output files written by the initial MR job are small, then a second MR
> job is run to merge the output into larger files (a copy from S3 to S3 in the
> current code).
> If the original output files are large enough, then the conditional task is
> followed by a move/rename which is very expensive in S3.
> We should keep staging data on HDFS previous to copying them to S3 as final
> files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14271) FileSinkOperator should not rename files to final paths when S3 is the default destination
[
https://issues.apache.org/jira/browse/HIVE-14271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abdullah Yousufi updated HIVE-14271:
Assignee: Sergio Peña (was: Abdullah Yousufi)
> FileSinkOperator should not rename files to final paths when S3 is the
> default destination
> --
>
> Key: HIVE-14271
> URL: https://issues.apache.org/jira/browse/HIVE-14271
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>
> FileSinkOperator does a rename of {{outPaths -> finalPaths}} when it finished
> writing all rows to a temporary path. The problem is that S3 does not support
> renaming.
> Two options can be considered:
> a. Use a copy operation instead. After FileSinkOperator writes all rows to
> outPaths, then the commit method will do a copy() call instead of move().
> b. Write row by row directly to the S3 path (see HIVE-1620). This may add
> better performance calls, but we should take care of the cleanup part in case
> of writing errors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14588) Add S3 credentials to the hidden configuration variable supported on HIVE-14207
[
https://issues.apache.org/jira/browse/HIVE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429454#comment-15429454
]
Hive QA commented on HIVE-14588:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12824690/HIVE-14588.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10470 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.service.cli.operation.TestOperationLoggingLayout.testSwitchLogLayout
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/949/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/949/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-949/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12824690 - PreCommit-HIVE-MASTER-Build
> Add S3 credentials to the hidden configuration variable supported on
> HIVE-14207
> ---
>
> Key: HIVE-14588
> URL: https://issues.apache.org/jira/browse/HIVE-14588
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Gabor Szadovszky
>Assignee: Gabor Szadovszky
> Attachments: HIVE-14588.patch
>
>
> Here's the list of S3 values we should hide:
> {code}
> fs.s3.awsAccessKeyId
> fs.s3.awsSecretAccessKey
> fs.s3n.awsAccessKeyId
> fs.s3n.awsSecretAccessKey
> fs.s3a.access.key
> fs.s3a.secret.key
> fs.s3a.proxy.password
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14502) Convert MiniTez tests to MiniLlap tests
[
https://issues.apache.org/jira/browse/HIVE-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429428#comment-15429428
]
Hive QA commented on HIVE-14502:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12824680/HIVE-14502.5.patch
{color:green}SUCCESS:{color} +1 due to 21 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 10473 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[alter_merge_2_orc]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[auto_join_filters]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[delete_orig_table]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llapdecider]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_vectorization_ppd]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[partition_column_names_with_leading_and_trailing_spaces]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[stats_filemetadata]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[tez_union_with_udf]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[union_type_chk]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[update_orig_table]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[vector_non_string_partition]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[vectorization_div0]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[vectorization_limit]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[insert1]
org.apache.hive.service.cli.operation.TestOperationLoggingLayout.testSwitchLogLayout
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/948/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/948/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-948/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12824680 - PreCommit-HIVE-MASTER-Build
> Convert MiniTez tests to MiniLlap tests
> ---
>
> Key: HIVE-14502
> URL: https://issues.apache.org/jira/browse/HIVE-14502
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14502.1.patch, HIVE-14502.2.patch,
> HIVE-14502.3.patch, HIVE-14502.4.patch, HIVE-14502.5.patch
>
>
> Llap shares most of the codepath with tez. MiniLlapCliDriver is much faster
> than MiniTezCliDriver because of threaded executors and caching.
> MiniTezCliDriver tests takes around 3hr 15mins to run around 400 tests. To
> cut down this test time significantly it makes sense to move over mive tez
> tests to mini llap tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14165) Remove Hive file listing during split computation
[
https://issues.apache.org/jira/browse/HIVE-14165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429389#comment-15429389
]
Hive QA commented on HIVE-14165:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12824670/HIVE-14165.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 61 failed/errored test(s), 10470 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_1_sql_std]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_const]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_precision2]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_opt_bucketing]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_evolved_parts]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[float_equality]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[select_dummy_source]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_literal]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_add_months]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_aes_decrypt]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_aes_encrypt]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitwise_shiftleft]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitwise_shiftright]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitwise_shiftrightunsigned]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bround]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_cbrt]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_chr]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_crc32]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_current_database]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_date_add]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_date_format]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_date_sub]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_decode]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_factorial]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_format_number]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_from_utc_timestamp]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_get_json_object]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_last_day]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_levenshtein]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_first_n]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_last_n]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_show_first_n]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_show_last_n]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_md5]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_months_between]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_quarter]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_replace]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_sha1]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_sha2]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_soundex]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_substring_index]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_to_utc_timestamp]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_trunc]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_version]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[hybridgrace_hashjoin_1]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[hybridgrace_hashjoin_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[select_dummy_source]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.beeline.TestBeeLineWithArgs.testConnectionWithURLParams
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadMissingPartitionBasicNeg[3]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs
org.apache.hive.jdbc.TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
org.apache.hive.jdbc.TestJdbcWithMiniLlap.testNonAsciiStrings
org.apache.hive.service.cli.operation.TestOperationLoggingLay
[jira] [Commented] (HIVE-14362) Support explain analyze in Hive
[
https://issues.apache.org/jira/browse/HIVE-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429347#comment-15429347
]
Hive QA commented on HIVE-14362:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12824671/HIVE-14362.02.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 10476 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_quoting]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_date]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_2]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[display_colstats_tbllvl]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exec_parallel_column_stats]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_display_colstats_tbllvl]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hadoop.hive.ql.exec.TestExplainTask.testExplainDoesSortMapValues
org.apache.hadoop.hive.ql.exec.TestExplainTask.testExplainDoesSortPathAsStrings
org.apache.hadoop.hive.ql.exec.TestExplainTask.testExplainDoesSortTopLevelMapEntries
org.apache.hive.service.cli.operation.TestOperationLoggingLayout.testSwitchLogLayout
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/946/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/946/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-946/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12824671 - PreCommit-HIVE-MASTER-Build
> Support explain analyze in Hive
> ---
>
> Key: HIVE-14362
> URL: https://issues.apache.org/jira/browse/HIVE-14362
> Project: Hive
> Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14362.01.patch, HIVE-14362.02.patch,
> compare_on_cluster.pdf
>
>
> Right now all the explain levels only support stats before query runs. We
> would like to have an explain analyze similar to Postgres for real stats
> after query runs. This will help to identify the major gap between
> estimated/real stats and make not only query optimization better but also
> query performance debugging easier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14590) CBO (Calcite Return Path) Incorrect result set when limit is present in one of the union branches
[
https://issues.apache.org/jira/browse/HIVE-14590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429318#comment-15429318
]
Hive QA commented on HIVE-14590:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12824661/HIVE-14590.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10444 tests
executed
*Failed tests:*
{noformat}
TestJdbcWithMiniKdcSQLAuthBinary - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_null]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
org.apache.hive.service.cli.operation.TestOperationLoggingLayout.testSwitchLogLayout
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/945/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/945/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-945/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12824661 - PreCommit-HIVE-MASTER-Build
> CBO (Calcite Return Path) Incorrect result set when limit is present in one
> of the union branches
> -
>
> Key: HIVE-14590
> URL: https://issues.apache.org/jira/browse/HIVE-14590
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-14590.patch
>
>
> Limit gets propagated outside union.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8960) ParsingException in the WHERE statement with a Sub Query
[ https://issues.apache.org/jira/browse/HIVE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429315#comment-15429315 ] Vikash Pareek commented on HIVE-8960: - I am trying following query, it is working in Impala but not in Hive. SELECT t1.col1 FROM table1 t1 LEFT OUTER JOIN table2 t2 ON (t1.col2 = t2.col2 AND t1.col3 = t2.col3) WHERE t2.col4 = (SELECT MAX(t22.col4) FROM table2 t22 WHERE t22.col4 <= t1.col4); Is there any alternative for this in Hive? > ParsingException in the WHERE statement with a Sub Query > > > Key: HIVE-8960 > URL: https://issues.apache.org/jira/browse/HIVE-8960 > Project: Hive > Issue Type: Bug > Components: Parser >Affects Versions: 0.13.0 > Environment: Secured HDP 2.1.3 with Hive 0.13.0 >Reporter: Rémy SAISSY > > Comparison with a Sub query in a WHERE statement does not work. > Given that id_chargement is an integer: > USE db1; > SELECT * FROM tbl1 a WHERE a.id_chargement > (SELECT MAX(b.id_chargement) > FROM tbl2 b); > or > SELECT * FROM tbl1 a WHERE a.id_chargement > (SELECT b.id_chargement FROM > tbl2 b LIMIT 1); > Both return the following parsing error: > Error: Error while compiling statement: FAILED: ParseException line 1:88 > cannot recognize input near 'SELECT' 'b' '.' in expression specification > (state=42000,code=4) > java.sql.SQLException: Error while compiling statement: FAILED: > ParseException line 1:88 cannot recognize input near 'SELECT' 'b' '.' in > expression specification > at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121) > at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109) > at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231) > at org.apache.hive.beeline.Commands.execute(Commands.java:736) > at org.apache.hive.beeline.Commands.sql(Commands.java:657) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:804) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:659) > at > org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:368) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:351) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13874) Tighten up EOF checking in Fast DeserializeRead classes; display better exception information; add new Unit Tests
[ https://issues.apache.org/jira/browse/HIVE-13874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13874: Resolution: Fixed Status: Resolved (was: Patch Available) > Tighten up EOF checking in Fast DeserializeRead classes; display better > exception information; add new Unit Tests > - > > Key: HIVE-13874 > URL: https://issues.apache.org/jira/browse/HIVE-13874 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 2.2.0, 2.1.1 > > Attachments: HIVE-13874.01.patch, HIVE-13874.02.patch, > HIVE-13874.03.patch, HIVE-13874.04.patch, HIVE-13874.05.patch > > > Tighten up EOF bounds checking in LazyBinaryDeserializeRead so bytes beyond > stated row end are never read. Use WritableUtils.decodeVIntSize to check for > room ahead like regular LazyBinary code does. > Display more detailed information when an exception is thrown by > DeserializeRead classes. > Add Unit Tests, including some designed that catch the errors like HIVE-13818. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13874) Tighten up EOF checking in Fast DeserializeRead classes; display better exception information; add new Unit Tests
[ https://issues.apache.org/jira/browse/HIVE-13874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429289#comment-15429289 ] Matt McCline commented on HIVE-13874: - Committed to master and branch-2.1 > Tighten up EOF checking in Fast DeserializeRead classes; display better > exception information; add new Unit Tests > - > > Key: HIVE-13874 > URL: https://issues.apache.org/jira/browse/HIVE-13874 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 2.2.0, 2.1.1 > > Attachments: HIVE-13874.01.patch, HIVE-13874.02.patch, > HIVE-13874.03.patch, HIVE-13874.04.patch, HIVE-13874.05.patch > > > Tighten up EOF bounds checking in LazyBinaryDeserializeRead so bytes beyond > stated row end are never read. Use WritableUtils.decodeVIntSize to check for > room ahead like regular LazyBinary code does. > Display more detailed information when an exception is thrown by > DeserializeRead classes. > Add Unit Tests, including some designed that catch the errors like HIVE-13818. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13874) Tighten up EOF checking in Fast DeserializeRead classes; display better exception information; add new Unit Tests
[ https://issues.apache.org/jira/browse/HIVE-13874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13874: Fix Version/s: 2.1.1 2.2.0 > Tighten up EOF checking in Fast DeserializeRead classes; display better > exception information; add new Unit Tests > - > > Key: HIVE-13874 > URL: https://issues.apache.org/jira/browse/HIVE-13874 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 2.2.0, 2.1.1 > > Attachments: HIVE-13874.01.patch, HIVE-13874.02.patch, > HIVE-13874.03.patch, HIVE-13874.04.patch, HIVE-13874.05.patch > > > Tighten up EOF bounds checking in LazyBinaryDeserializeRead so bytes beyond > stated row end are never read. Use WritableUtils.decodeVIntSize to check for > room ahead like regular LazyBinary code does. > Display more detailed information when an exception is thrown by > DeserializeRead classes. > Add Unit Tests, including some designed that catch the errors like HIVE-13818. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13874) Tighten up EOF checking in Fast DeserializeRead classes; display better exception information; add new Unit Tests
[ https://issues.apache.org/jira/browse/HIVE-13874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429287#comment-15429287 ] Matt McCline commented on HIVE-13874: - Test failures are unrelated. > Tighten up EOF checking in Fast DeserializeRead classes; display better > exception information; add new Unit Tests > - > > Key: HIVE-13874 > URL: https://issues.apache.org/jira/browse/HIVE-13874 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-13874.01.patch, HIVE-13874.02.patch, > HIVE-13874.03.patch, HIVE-13874.04.patch, HIVE-13874.05.patch > > > Tighten up EOF bounds checking in LazyBinaryDeserializeRead so bytes beyond > stated row end are never read. Use WritableUtils.decodeVIntSize to check for > room ahead like regular LazyBinary code does. > Display more detailed information when an exception is thrown by > DeserializeRead classes. > Add Unit Tests, including some designed that catch the errors like HIVE-13818. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13874) Tighten up EOF checking in Fast DeserializeRead classes; display better exception information; add new Unit Tests
[
https://issues.apache.org/jira/browse/HIVE-13874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429286#comment-15429286
]
Hive QA commented on HIVE-13874:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12824660/HIVE-13874.05.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10470 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.service.cli.operation.TestOperationLoggingLayout.testSwitchLogLayout
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/944/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/944/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-944/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12824660 - PreCommit-HIVE-MASTER-Build
> Tighten up EOF checking in Fast DeserializeRead classes; display better
> exception information; add new Unit Tests
> -
>
> Key: HIVE-13874
> URL: https://issues.apache.org/jira/browse/HIVE-13874
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-13874.01.patch, HIVE-13874.02.patch,
> HIVE-13874.03.patch, HIVE-13874.04.patch, HIVE-13874.05.patch
>
>
> Tighten up EOF bounds checking in LazyBinaryDeserializeRead so bytes beyond
> stated row end are never read. Use WritableUtils.decodeVIntSize to check for
> room ahead like regular LazyBinary code does.
> Display more detailed information when an exception is thrown by
> DeserializeRead classes.
> Add Unit Tests, including some designed that catch the errors like HIVE-13818.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
