[jira] [Commented] (HIVE-9780) Add another level of explain for RDBMS audience
[ https://issues.apache.org/jira/browse/HIVE-9780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015391#comment-15015391 ] Lefty Leverenz commented on HIVE-9780: -- On by default? But it was set to false originally and it's still set to false in 1.2.1. The documentation also says it's false by default: "After set hive.explain.user=true (default is false) if the following query is sent, the user can see a much more clearly readable tree of operations." > Add another level of explain for RDBMS audience > --- > > Key: HIVE-9780 > URL: https://issues.apache.org/jira/browse/HIVE-9780 > Project: Hive > Issue Type: New Feature > Components: Diagnosability >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong >Priority: Minor > Fix For: 1.2.0 > > Attachments: HIVE-9780.01.patch, HIVE-9780.02.patch, > HIVE-9780.03.patch, HIVE-9780.04.patch, HIVE-9780.05.patch, > HIVE-9780.06.patch, HIVE-9780.07.patch, HIVE-9780.08.patch, > HIVE-9780.09.patch, HIVE-9780.10.patch > > > Current Hive Explain (default) is targeted at MR Audience. We need a new > level of explain plan to be targeted at RDBMS audience. The explain requires > these: > 1) The focus needs to be on what part of the query is being executed rather > than internals of the engines > 2) There needs to be a clearly readable tree of operations > 3) Examples - Table scan should mention the table being scanned, the Sarg, > the size of table and expected cardinality after the Sarg'ed read. The join > should mention the table being joined with and the join condition. The > aggregate should mention the columns in the group-by. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12389) CompactionTxnHandler.cleanEmptyAbortedTxns() should safeguard against huge IN clauses
[
https://issues.apache.org/jira/browse/HIVE-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015396#comment-15015396
]
Hive QA commented on HIVE-12389:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12772911/HIVE-12389.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 9827 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterPartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterViewParititon
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testColumnStatistics
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testComplexTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testComplexTypeApi
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testConcurrentMetastores
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDBOwner
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDBOwnerChange
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDatabase
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDatabaseLocation
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDatabaseLocationWithPermissionProblems
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDropTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testFilterLastPartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testFilterSinglePartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testFunctionWithResources
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testGetConfigValue
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testGetTableObjects
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testListPartitionNames
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testListPartitions
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testNameMethods
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartitionFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testRenamePartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testRetriableClientWithConnLifetime
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleTypeApi
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testStatsFastTrivial
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSynchronized
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTableDatabase
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTableFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testValidateTableCols
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.jdbc.miniHS2.TestHs2Metrics.testMetrics
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6079/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6079/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6079/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 38 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12772911 - PreCommit-HIVE-TRUNK-Build
> CompactionTxnHandler.cleanEmptyAbortedTxns() should safeguard against huge IN
> clauses
> -
>
> Key: HIVE-12389
> URL: https://issues.apache.org/jira/browse/HIVE-12389
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-12389.2.patch, HIVE-12389.patch
>
>
> in extreme situations, due to misconfigurations, it may be possible to have
> 100Ks or even 1Ms of aborted txns.
> This causes delete from TXNS where txn_id in (...) to have a huge IN clause
> and DB chokes.
> Should use something like TxnHandler.TIMED_OUT_TXN_ABORT_BATCH_SIZE to b
[jira] [Commented] (HIVE-9780) Add another level of explain for RDBMS audience
[ https://issues.apache.org/jira/browse/HIVE-9780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015416#comment-15015416 ] Pengcheng Xiong commented on HIVE-9780: --- [~leftylev], yes, it is on by default since HIVE-11037. Thanks. > Add another level of explain for RDBMS audience > --- > > Key: HIVE-9780 > URL: https://issues.apache.org/jira/browse/HIVE-9780 > Project: Hive > Issue Type: New Feature > Components: Diagnosability >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong >Priority: Minor > Fix For: 1.2.0 > > Attachments: HIVE-9780.01.patch, HIVE-9780.02.patch, > HIVE-9780.03.patch, HIVE-9780.04.patch, HIVE-9780.05.patch, > HIVE-9780.06.patch, HIVE-9780.07.patch, HIVE-9780.08.patch, > HIVE-9780.09.patch, HIVE-9780.10.patch > > > Current Hive Explain (default) is targeted at MR Audience. We need a new > level of explain plan to be targeted at RDBMS audience. The explain requires > these: > 1) The focus needs to be on what part of the query is being executed rather > than internals of the engines > 2) There needs to be a clearly readable tree of operations > 3) Examples - Table scan should mention the table being scanned, the Sarg, > the size of table and expected cardinality after the Sarg'ed read. The join > should mention the table being joined with and the join condition. The > aggregate should mention the columns in the group-by. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12367) Lock/unlock database should add current database to inputs and outputs of authz hook
[
https://issues.apache.org/jira/browse/HIVE-12367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015432#comment-15015432
]
Dapeng Sun commented on HIVE-12367:
---
I think the lock should still be {{DDL_NO_LOCK}}, I misunderstand LOCK and
DDL_LOCK...
> Lock/unlock database should add current database to inputs and outputs of
> authz hook
>
>
> Key: HIVE-12367
> URL: https://issues.apache.org/jira/browse/HIVE-12367
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Affects Versions: 1.2.1
>Reporter: Dapeng Sun
>Assignee: Dapeng Sun
> Attachments: HIVE-12367.001.patch, HIVE-12367.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12393) Simplify ColumnPruner when CBO optimizes the query
[ https://issues.apache.org/jira/browse/HIVE-12393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015461#comment-15015461 ] Jesus Camacho Rodriguez commented on HIVE-12393: It does not need to be included in 2.0.0. More work needs to be done... > Simplify ColumnPruner when CBO optimizes the query > -- > > Key: HIVE-12393 > URL: https://issues.apache.org/jira/browse/HIVE-12393 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > > The plan for any given query optimized by CBO will always contain a Project > operator on top of the TS that prunes that columns that are not needed. > Thus, there is no need for Hive optimizer to traverse the whole plan to check > which columns can be pruned. In fact, Hive ColumnPruner optimizer only needs > to match TS operators when CBO optimized the plan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-12479: --- Description: The row-counts expected with and without vectorization differ. The attached small-scale repro case produces 5 rows with vectorized multi-key joins and 53 rows without the vectorized join. > Vectorization: vectorized MultiKeyJoins with 2 integers > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-12479: --- Attachment: HIVE-12479.tar.gz > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-12479: --- Summary: Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle (was: Vectorization: vectorized MultiKeyJoins with 2 integers) > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015538#comment-15015538 ] Gopal V commented on HIVE-12479: [~mmccline]: The attached data-set has 2 ORC input files and the query to run. TPC-DS is entirely single integer joins - so can't really port the example problem onto TPC-DS. > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12184) DESCRIBE of fully qualified table fails when db and table name match and non-default database is in use
[
https://issues.apache.org/jira/browse/HIVE-12184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015632#comment-15015632
]
Hive QA commented on HIVE-12184:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12772924/HIVE-12184.8.patch
{color:green}SUCCESS:{color} +1 due to 23 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 9832 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6080/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6080/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6080/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12772924 - PreCommit-HIVE-TRUNK-Build
> DESCRIBE of fully qualified table fails when db and table name match and
> non-default database is in use
> ---
>
> Key: HIVE-12184
> URL: https://issues.apache.org/jira/browse/HIVE-12184
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.2.1
>Reporter: Lenni Kuff
>Assignee: Naveen Gangam
> Attachments: HIVE-12184.2.patch, HIVE-12184.3.patch,
> HIVE-12184.4.patch, HIVE-12184.5.patch, HIVE-12184.6.patch,
> HIVE-12184.7.patch, HIVE-12184.8.patch, HIVE-12184.patch
>
>
> DESCRIBE of fully qualified table fails when db and table name match and
> non-default database is in use.
> Repro:
> {code}
> : jdbc:hive2://localhost:1/default> create database foo;
> No rows affected (0.116 seconds)
> 0: jdbc:hive2://localhost:1/default> create table foo.foo(i int);
> 0: jdbc:hive2://localhost:1/default> describe foo.foo;
> +---++--+--+
> | col_name | data_type | comment |
> +---++--+--+
> | i | int| |
> +---++--+--+
> 1 row selected (0.049 seconds)
> 0: jdbc:hive2://localhost:1/default> use foo;
> 0: jdbc:hive2://localhost:1/default> describe foo.foo;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Error in getting fields from
> serde.Invalid Field foo (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11527) bypass HiveServer2 thrift interface for query results
[ https://issues.apache.org/jira/browse/HIVE-11527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015634#comment-15015634 ] Takanobu Asanuma commented on HIVE-11527: - I'd like to share my thoughts for this work. I'm going to add a new thrift api to get a uri of the result data. New data flow is like below: 1. When JDBC client calls HiveQueryResultSet#next(), JDBC driver calls the new thrift API. 2. HiveServer2 returns the path of the result data. IIUC, FetchWork has the uri. NOTE: Like "select * from tablename limit 10", some queries don't run MR/Tez/Spark jobs and the uri is different from the final data. In this case, we use current implementation. 3. JDBC driver gets the uri and downloads the data by WebHDFS. 4. JDBC driver decodes the data and creates RowSet. I'm writing codes and I will upload a wip patch next week. If you have any thoughts on this jira, please share it with me. > bypass HiveServer2 thrift interface for query results > - > > Key: HIVE-11527 > URL: https://issues.apache.org/jira/browse/HIVE-11527 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Sergey Shelukhin >Assignee: Takanobu Asanuma > > Right now, HS2 reads query results and returns them to the caller via its > thrift API. > There should be an option for HS2 to return some pointer to results (an HDFS > link?) and for the user to read the results directly off HDFS inside the > cluster, or via something like WebHDFS outside the cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12382) return actual row count for JDBC executeUpdate
[ https://issues.apache.org/jira/browse/HIVE-12382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15016713#comment-15016713 ] Nemon Lou commented on HIVE-12382: -- Hive counters for "RECORDS_OUT" is wrong in some situation as i linked. > return actual row count for JDBC executeUpdate > -- > > Key: HIVE-12382 > URL: https://issues.apache.org/jira/browse/HIVE-12382 > Project: Hive > Issue Type: Improvement > Components: JDBC >Reporter: Nemon Lou >Assignee: Nemon Lou >Priority: Minor > > when running sql like 'insert into/overwrite table', > user may want to know how many rows are inserted . > Return actual row count for HiveStatement.executeUpdate is useful in such > cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12184) DESCRIBE of fully qualified table fails when db and table name match and non-default database is in use
[
https://issues.apache.org/jira/browse/HIVE-12184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018028#comment-15018028
]
Naveen Gangam commented on HIVE-12184:
--
The 2 failures are not related to the patch and seem to fail in lot of other
builds too.
> DESCRIBE of fully qualified table fails when db and table name match and
> non-default database is in use
> ---
>
> Key: HIVE-12184
> URL: https://issues.apache.org/jira/browse/HIVE-12184
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.2.1
>Reporter: Lenni Kuff
>Assignee: Naveen Gangam
> Attachments: HIVE-12184.2.patch, HIVE-12184.3.patch,
> HIVE-12184.4.patch, HIVE-12184.5.patch, HIVE-12184.6.patch,
> HIVE-12184.7.patch, HIVE-12184.8.patch, HIVE-12184.patch
>
>
> DESCRIBE of fully qualified table fails when db and table name match and
> non-default database is in use.
> Repro:
> {code}
> : jdbc:hive2://localhost:1/default> create database foo;
> No rows affected (0.116 seconds)
> 0: jdbc:hive2://localhost:1/default> create table foo.foo(i int);
> 0: jdbc:hive2://localhost:1/default> describe foo.foo;
> +---++--+--+
> | col_name | data_type | comment |
> +---++--+--+
> | i | int| |
> +---++--+--+
> 1 row selected (0.049 seconds)
> 0: jdbc:hive2://localhost:1/default> use foo;
> 0: jdbc:hive2://localhost:1/default> describe foo.foo;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Error in getting fields from
> serde.Invalid Field foo (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11684) Implement limit pushdown through outer join in CBO
[
https://issues.apache.org/jira/browse/HIVE-11684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018096#comment-15018096
]
Hive QA commented on HIVE-11684:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12772967/HIVE-11684.21.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9833 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_index_bitmap_auto
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6081/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6081/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6081/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12772967 - PreCommit-HIVE-TRUNK-Build
> Implement limit pushdown through outer join in CBO
> --
>
> Key: HIVE-11684
> URL: https://issues.apache.org/jira/browse/HIVE-11684
> Project: Hive
> Issue Type: New Feature
> Components: CBO
>Affects Versions: 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-11684.01.patch, HIVE-11684.02.patch,
> HIVE-11684.03.patch, HIVE-11684.04.patch, HIVE-11684.05.patch,
> HIVE-11684.07.patch, HIVE-11684.08.patch, HIVE-11684.09.patch,
> HIVE-11684.10.patch, HIVE-11684.11.patch, HIVE-11684.12.patch,
> HIVE-11684.12.patch, HIVE-11684.14.patch, HIVE-11684.15.patch,
> HIVE-11684.16.patch, HIVE-11684.17.patch, HIVE-11684.18.patch,
> HIVE-11684.19.patch, HIVE-11684.20.patch, HIVE-11684.21.patch,
> HIVE-11684.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12482) When execution.engine=tez,set mapreduce.job.name does not work.
[ https://issues.apache.org/jira/browse/HIVE-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaowei Wang updated HIVE-12482: Attachment: HIVE-12482.0.patch > When execution.engine=tez,set mapreduce.job.name does not work. > --- > > Key: HIVE-12482 > URL: https://issues.apache.org/jira/browse/HIVE-12482 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 0.14.0, 1.0.0, 1.0.1, 1.2.1 >Reporter: Xiaowei Wang > Fix For: 0.14.1 > > Attachments: HIVE-12482.0.patch > > > When execution.engine=tez,set mapreduce.job.name does not work. > In Tez mode, the default job name is "Hive_"+Sessionid ,for example > HIVE-ce5784d0-320c-4fb9-8b0b-2d92539dfd9e .It is difficulty to distinguish > job when there are too much jobs . > A better way is to set the var of mapreduce.job.name .But set > mapreduce.job.name does not work! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12469) Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address vulnerability
[
https://issues.apache.org/jira/browse/HIVE-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018140#comment-15018140
]
Dmitry Tolpeko commented on HIVE-12469:
---
This dependency can removed, not sure now how it was appeared, sorry. I ran
tests successfully.
> Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address
> vulnerability
> -
>
> Key: HIVE-12469
> URL: https://issues.apache.org/jira/browse/HIVE-12469
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Reuben Kuhnert
>Assignee: Reuben Kuhnert
>Priority: Blocker
>
> Currently the commons-collections (3.2.1) library allows for invocation of
> arbitrary code through {{InvokerTransformer}}, need to bump the version of
> commons-collections from 3.2.1 to 3.2.2 to resolve this issue.
> Results of {{mvn dependency:tree}}:
> {code}
> [INFO]
>
> [INFO] Building Hive HPL/SQL 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-hplsql ---
> [INFO] org.apache.hive:hive-hplsql:jar:2.0.0-SNAPSHOT
> [INFO] +- com.google.guava:guava:jar:14.0.1:compile
> [INFO] +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Packaging 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.hive:hive-hbase-handler:jar:2.0.0-SNAPSHOT:compile
> [INFO] | +- org.apache.hbase:hbase-server:jar:1.1.1:compile
> [INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Common 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-common ---
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {{Hadoop-Common}} dependency also found in: LLAP, Serde, Storage, Shims,
> Shims Common, Shims Scheduler)
> {code}
> [INFO]
>
> [INFO] Building Hive Ant Utilities 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-ant ---
> [INFO] | +- commons-collections:commons-collections:jar:3.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO]
>
> [INFO] Building Hive Accumulo Handler 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.accumulo:accumulo-core:jar:1.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12482) When execution.engine=tez,set mapreduce.job.name does not work.
[ https://issues.apache.org/jira/browse/HIVE-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018148#comment-15018148 ] Xiaowei Wang commented on HIVE-12482: - [~ashutoshc] [~ashutoshc] [~xuefuz] [~gopalv] > When execution.engine=tez,set mapreduce.job.name does not work. > --- > > Key: HIVE-12482 > URL: https://issues.apache.org/jira/browse/HIVE-12482 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 0.14.0, 1.0.0, 1.0.1, 1.2.1 >Reporter: Xiaowei Wang > Fix For: 0.14.1 > > Attachments: HIVE-12482.0.patch > > > When execution.engine=tez,set mapreduce.job.name does not work. > In Tez mode, the default job name is "Hive_"+Sessionid ,for example > HIVE-ce5784d0-320c-4fb9-8b0b-2d92539dfd9e .It is difficulty to distinguish > job when there are too much jobs . > A better way is to set the var of mapreduce.job.name .But set > mapreduce.job.name does not work! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11371) Null pointer exception for nested table query when using ORC versus text
[ https://issues.apache.org/jira/browse/HIVE-11371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-11371: Attachment: HIVE-11371.01.patch > Null pointer exception for nested table query when using ORC versus text > > > Key: HIVE-11371 > URL: https://issues.apache.org/jira/browse/HIVE-11371 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.2.0 >Reporter: N Campbell >Assignee: Matt McCline > Attachments: HIVE-11371.01.patch, TJOIN1, TJOIN2, TJOIN3, TJOIN4 > > > Following query will fail if the file format is ORC > select tj1rnum, tj2rnum, tjoin3.rnum as rnumt3 from (select tjoin1.rnum > tj1rnum, tjoin2.rnum tj2rnum, tjoin2.c1 tj2c1 from tjoin1 left outer join > tjoin2 on tjoin1.c1 = tjoin2.c1 ) tj left outer join tjoin3 on tj2c1 = > tjoin3.c1 > aused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.vector.VectorCopyRow$LongCopyRow.copy(VectorCopyRow.java:60) > at > org.apache.hadoop.hive.ql.exec.vector.VectorCopyRow.copyByReference(VectorCopyRow.java:260) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultMultiValue(VectorMapJoinGenerateResultOperator.java:238) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinOuterGenerateResultOperator.finishOuter(VectorMapJoinOuterGenerateResultOperator.java:495) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinOuterLongOperator.process(VectorMapJoinOuterLongOperator.java:430) > ... 22 more > ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 > killedTasks:0, Vertex vertex_1437788144883_0004_2_02 [Map 1] killed/failed > due to:null]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 > killedVertices:0 > SQLState: 08S01 > ErrorCode: 2 > getDatabaseProductNameApache Hive > getDatabaseProductVersion 1.2.1.2.3.0.0-2557 > getDriverName Hive JDBC > getDriverVersion 1.2.1.2.3.0.0-2557 > getDriverMajorVersion 1 > getDriverMinorVersion 2 > create table if not exists TJOIN1 (RNUM int , C1 int, C2 int) > -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' > STORED AS orc; > create table if not exists TJOIN2 (RNUM int , C1 int, C2 char(2)) > -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' > STORED AS orc ; > create table if not exists TJOIN3 (RNUM int , C1 int, C2 char(2)) > -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' > STORED AS orc ; > create table if not exists TJOIN4 (RNUM int , C1 int, C2 char(2)) > -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' > STORED AS orc ; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11684) Implement limit pushdown through outer join in CBO
[ https://issues.apache.org/jira/browse/HIVE-11684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018278#comment-15018278 ] Jesus Camacho Rodriguez commented on HIVE-11684: [~jpullokkaran], fails are not related, could you take a look so it can go in 2.0.0? Thanks > Implement limit pushdown through outer join in CBO > -- > > Key: HIVE-11684 > URL: https://issues.apache.org/jira/browse/HIVE-11684 > Project: Hive > Issue Type: New Feature > Components: CBO >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-11684.01.patch, HIVE-11684.02.patch, > HIVE-11684.03.patch, HIVE-11684.04.patch, HIVE-11684.05.patch, > HIVE-11684.07.patch, HIVE-11684.08.patch, HIVE-11684.09.patch, > HIVE-11684.10.patch, HIVE-11684.11.patch, HIVE-11684.12.patch, > HIVE-11684.12.patch, HIVE-11684.14.patch, HIVE-11684.15.patch, > HIVE-11684.16.patch, HIVE-11684.17.patch, HIVE-11684.18.patch, > HIVE-11684.19.patch, HIVE-11684.20.patch, HIVE-11684.21.patch, > HIVE-11684.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12456) QueryId can't be stored in the configuration of the SessionState since multiple queries can run in a single session
[ https://issues.apache.org/jira/browse/HIVE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12456: Attachment: (was: HIVE-12456.patch) > QueryId can't be stored in the configuration of the SessionState since > multiple queries can run in a single session > --- > > Key: HIVE-12456 > URL: https://issues.apache.org/jira/browse/HIVE-12456 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > > Follow up on HIVE-11488 which stores the queryId in the sessionState conf. If > multiple queries run at the same time, then the logging will get wrong > queryId from the sessionState. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12456) QueryId can't be stored in the configuration of the SessionState since multiple queries can run in a single session
[ https://issues.apache.org/jira/browse/HIVE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12456: Attachment: HIVE-12456.patch > QueryId can't be stored in the configuration of the SessionState since > multiple queries can run in a single session > --- > > Key: HIVE-12456 > URL: https://issues.apache.org/jira/browse/HIVE-12456 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-12456.patch > > > Follow up on HIVE-11488 which stores the queryId in the sessionState conf. If > multiple queries run at the same time, then the logging will get wrong > queryId from the sessionState. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12483) Fix precommit Spark test branch
[ https://issues.apache.org/jira/browse/HIVE-12483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-12483: --- Attachment: HIVE-12483.1-spark.patch > Fix precommit Spark test branch > --- > > Key: HIVE-12483 > URL: https://issues.apache.org/jira/browse/HIVE-12483 > Project: Hive > Issue Type: Task >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-12483.1-spark.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-6113) Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
[ https://issues.apache.org/jira/browse/HIVE-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksiy Sayankin updated HIVE-6113: --- Attachment: HIVE-6113.3.patch > Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient > -- > > Key: HIVE-6113 > URL: https://issues.apache.org/jira/browse/HIVE-6113 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.1 > Environment: hadoop-0.20.2-cdh3u3,hive-0.12.0 >Reporter: William Stone >Assignee: Oleksiy Sayankin >Priority: Critical > Labels: HiveMetaStoreClient, metastore, unable_instantiate > Attachments: HIVE-6113-2.patch, HIVE-6113.3.patch, HIVE-6113.patch > > > When I exccute SQL "use fdm; desc formatted fdm.tableName;" in python, throw > Error as followed. > but when I tryit again , It will success. > 2013-12-25 03:01:32,290 ERROR exec.DDLTask (DDLTask.java:execute(435)) - > org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: > Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1143) > at > org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1128) > at > org.apache.hadoop.hive.ql.exec.DDLTask.switchDatabase(DDLTask.java:3479) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:237) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1414) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1192) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1020) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:260) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:217) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:507) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:875) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:769) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:708) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:197) > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1217) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:62) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72) > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2372) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2383) > at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1139) > ... 20 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1210) > ... 25 more > Caused by: javax.jdo.JDODataStoreException: Exception thrown flushing changes > to datastore > NestedThrowables: > java.sql.BatchUpdateException: Duplicate entry 'default' for key > 'UNIQUE_DATABASE' > at > org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451) > at > org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165) > at > org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:358) > at > org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:404) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java
[jira] [Commented] (HIVE-12017) Do not disable CBO by default when number of joins in a query is equal or less than 1
[
https://issues.apache.org/jira/browse/HIVE-12017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018414#comment-15018414
]
Hive QA commented on HIVE-12017:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12773281/HIVE-12017.13.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 9831 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_cross_product_check_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynamic_rdd_cache
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join42
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join43
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_runtime_skewjoin_mapjoin_spark
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_table_access_keys_stats
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_inner_join
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join0
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_sortmerge_join_12
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cross_product_check_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cross_product_check_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mergejoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_auto_smb_mapjoin_14
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_12
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_cross_product_check_1
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_cross_product_check_2
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dynamic_rdd_cache
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join_star
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_mergejoins_mixed
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_table_access_keys_stats
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6082/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6082/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6082/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 30 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12773281 - PreCommit-HIVE-TRUNK-Build
> Do not disable CBO by default when number of joins in a query is equal or
> less than 1
> -
>
> Key: HIVE-12017
> URL: https://issues.apache.org/jira/browse/HIVE-12017
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12017.01.patch, HIVE-12017.02.patch,
> HIVE-12017.03.patch, HIVE-12017.04.patch, HIVE-12017.05.patch,
> HIVE-12017.06.patch, HIVE-12017.07.patch, HIVE-12017.08.patch,
> HIVE-12017.09.patch, HIVE-12017.10.patch, HIVE-12017.11.patch,
> HIVE-12017.12.patch, HIVE-12017.13.patch
>
>
> Instead, we could disable some parts of CBO that are not relevant if the
> query contains 1 or 0 joins. Implementation should be able to define easily
> other query patterns for which we might disable some parts of CBO (in case we
> want to do it in the future).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018448#comment-15018448 ] Szehon Ho commented on HIVE-12406: -- +1 lgtm. Minor suggestion is to annotate the new class with public and stability interfaces to make it clear-er in future. > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12476) Metastore NPE on Oracle with Direct SQL
[
https://issues.apache.org/jira/browse/HIVE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018460#comment-15018460
]
Sushanth Sowmyan commented on HIVE-12476:
-
Good catch, Jason, thanks for the update. +1.
It looks like I'd missed these when I worked on HIVE-8485. That said, I now
wonder how we could proactively find other cases where we might have a similar
problem that we've not yet realized exist(I verified now that master has only 3
.putToParameters calls, all of which you've changed) - it's also possible for
new code being introduced to not use this without realizing the impact, and
since our test framework runs mainly on Derby and sometimes MySQL, it's quite
possible for us to miss it and have to patch it later.
> Metastore NPE on Oracle with Direct SQL
> ---
>
> Key: HIVE-12476
> URL: https://issues.apache.org/jira/browse/HIVE-12476
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-12476.1.patch
>
>
> Stack trace looks very similar to HIVE-8485. I believe the metastore's Direct
> SQL mode requires additional fixes similar to HIVE-8485, around the
> Partition/StorageDescriptorSerDe parameters.
> {noformat}
> 2015-11-19 18:08:33,841 ERROR [pool-5-thread-2]: server.TThreadPoolServer
> (TThreadPoolServer.java:run(296)) - Error occurred during processing of
> message.
> java.lang.NullPointerException
> at
> org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:200)
> at
> org.apache.hadoop.hive.metastore.api.SerDeInfo$SerDeInfoStandardScheme.write(SerDeInfo.java:579)
> at
> org.apache.hadoop.hive.metastore.api.SerDeInfo$SerDeInfoStandardScheme.write(SerDeInfo.java:501)
> at
> org.apache.hadoop.hive.metastore.api.SerDeInfo.write(SerDeInfo.java:439)
> at
> org.apache.hadoop.hive.metastore.api.StorageDescriptor$StorageDescriptorStandardScheme.write(StorageDescriptor.java:1490)
> at
> org.apache.hadoop.hive.metastore.api.StorageDescriptor$StorageDescriptorStandardScheme.write(StorageDescriptor.java:1288)
> at
> org.apache.hadoop.hive.metastore.api.StorageDescriptor.write(StorageDescriptor.java:1154)
> at
> org.apache.hadoop.hive.metastore.api.Partition$PartitionStandardScheme.write(Partition.java:1072)
> at
> org.apache.hadoop.hive.metastore.api.Partition$PartitionStandardScheme.write(Partition.java:929)
> at
> org.apache.hadoop.hive.metastore.api.Partition.write(Partition.java:825)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_result$get_partitions_resultStandardScheme.write(ThriftHiveMetastore.java:64470)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_result$get_partitions_resultStandardScheme.write(ThriftHiveMetastore.java:64402)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_result.write(ThriftHiveMetastore.java:64340)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:53)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:681)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:676)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:676)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12437) SMB join in tez fails when one of the tables is empty
[ https://issues.apache.org/jira/browse/HIVE-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018472#comment-15018472 ] Vikram Dixit K commented on HIVE-12437: --- [~sershe] I think the only test failure of concern is the TestMiniLlapCliDriver. Should I be running a specific test here? > SMB join in tez fails when one of the tables is empty > - > > Key: HIVE-12437 > URL: https://issues.apache.org/jira/browse/HIVE-12437 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 1.0.1, 1.2.1 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K >Priority: Critical > Attachments: HIVE-12437.1.patch, HIVE-12437.2.patch > > > It looks like a better check for empty tables is to depend on the existence > of the record reader for the input from tez. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12437) SMB join in tez fails when one of the tables is empty
[ https://issues.apache.org/jira/browse/HIVE-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018493#comment-15018493 ] Sergey Shelukhin commented on HIVE-12437: - It looks like the test failed entirely, before even initialization. That is suspicious... you can try locally or resubmit the patch. > SMB join in tez fails when one of the tables is empty > - > > Key: HIVE-12437 > URL: https://issues.apache.org/jira/browse/HIVE-12437 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 1.0.1, 1.2.1 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K >Priority: Critical > Attachments: HIVE-12437.1.patch, HIVE-12437.2.patch > > > It looks like a better check for empty tables is to depend on the existence > of the record reader for the input from tez. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12437) SMB join in tez fails when one of the tables is empty
[ https://issues.apache.org/jira/browse/HIVE-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018495#comment-15018495 ] Sergey Shelukhin commented on HIVE-12437: - I see it also happens on other JIRAs... would need to see if something broke it > SMB join in tez fails when one of the tables is empty > - > > Key: HIVE-12437 > URL: https://issues.apache.org/jira/browse/HIVE-12437 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 1.0.1, 1.2.1 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K >Priority: Critical > Attachments: HIVE-12437.1.patch, HIVE-12437.2.patch > > > It looks like a better check for empty tables is to depend on the existence > of the record reader for the input from tez. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12319) Remove HadoopShims::getHadoopConfNames()
[ https://issues.apache.org/jira/browse/HIVE-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018504#comment-15018504 ] Sergey Shelukhin commented on HIVE-12319: - This patch has broken MiniLlap test - it has failed in the QA ran above and it has failed in the same way in every run starting with http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6078/ , the first build with this patch. Can you please fix or revert? By default, I will revert after lunch (in 2.5 hours or so) > Remove HadoopShims::getHadoopConfNames() > > > Key: HIVE-12319 > URL: https://issues.apache.org/jira/browse/HIVE-12319 > Project: Hive > Issue Type: Improvement > Components: Shims >Affects Versions: 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Aleksei Statkevich > Fix For: 2.0.0 > > Attachments: HIVE-12319.1.patch, HIVE-12319.patch > > > It was introduced in HIVE-6159 It has served its purpose now that we support > only Hadoop 2.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12406: Attachment: HIVE-12406.2.patch > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018519#comment-15018519 ] Aihua Xu commented on HIVE-12406: - Thanks [~szehon] for reviewing. I attached the new patch to address the comments. > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12175) Upgrade Kryo version to 3.0.x
[
https://issues.apache.org/jira/browse/HIVE-12175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-12175:
-
Attachment: HIVE-12175.6.patch
Rebased patch to latest master
> Upgrade Kryo version to 3.0.x
> -
>
> Key: HIVE-12175
> URL: https://issues.apache.org/jira/browse/HIVE-12175
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 2.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-12175.1.patch, HIVE-12175.2.patch,
> HIVE-12175.3.patch, HIVE-12175.3.patch, HIVE-12175.4.patch,
> HIVE-12175.5.patch, HIVE-12175.6.patch
>
>
> Current version of kryo (2.22) has some issue (refer exception below and in
> HIVE-12174) with serializing ArrayLists generated using Arrays.asList(). We
> need to either replace all occurrences of Arrays.asList() or change the
> current StdInstantiatorStrategy. This issue is fixed in later versions and
> kryo community recommends using DefaultInstantiatorStrategy with fallback to
> StdInstantiatorStrategy. More discussion about this issue is here
> https://github.com/EsotericSoftware/kryo/issues/216. Alternatively, custom
> serilization/deserilization class can be provided for Arrays.asList.
> Also, kryo 3.0 introduced unsafe based serialization which claims to have
> much better performance for certain types of serialization.
> Exception:
> {code}
> Caused by: java.lang.NullPointerException
> at java.util.Arrays$ArrayList.size(Arrays.java:2847)
> at java.util.AbstractList.add(AbstractList.java:108)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> ... 57 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12338) Add webui to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018537#comment-15018537 ] Mohit Sabharwal commented on HIVE-12338: LGTM. +1 (non-binding) > Add webui to HiveServer2 > > > Key: HIVE-12338 > URL: https://issues.apache.org/jira/browse/HIVE-12338 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-12338.1.patch, hs2-conf.png, hs2-logs.png, > hs2-metrics.png, hs2-webui.png > > > A web ui for HiveServer2 can show some useful information such as: > > 1. Sessions, > 2. Queries that are executing on the HS2, their states, starting time, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12469) Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address vulnerability
[
https://issues.apache.org/jira/browse/HIVE-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-12469:
Attachment: HIVE-12469.patch
I have removed direct dependency. For transitive dependecies, I have declared
the scope to provided to ensure its not in runtime classpath. [~sircodesalot]
[~dmtolpeko] would you guys like to take a look?
> Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address
> vulnerability
> -
>
> Key: HIVE-12469
> URL: https://issues.apache.org/jira/browse/HIVE-12469
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
>Reporter: Reuben Kuhnert
>Assignee: Reuben Kuhnert
>Priority: Blocker
> Attachments: HIVE-12469.patch
>
>
> Currently the commons-collections (3.2.1) library allows for invocation of
> arbitrary code through {{InvokerTransformer}}, need to bump the version of
> commons-collections from 3.2.1 to 3.2.2 to resolve this issue.
> Results of {{mvn dependency:tree}}:
> {code}
> [INFO]
>
> [INFO] Building Hive HPL/SQL 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-hplsql ---
> [INFO] org.apache.hive:hive-hplsql:jar:2.0.0-SNAPSHOT
> [INFO] +- com.google.guava:guava:jar:14.0.1:compile
> [INFO] +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Packaging 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.hive:hive-hbase-handler:jar:2.0.0-SNAPSHOT:compile
> [INFO] | +- org.apache.hbase:hbase-server:jar:1.1.1:compile
> [INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Common 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-common ---
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {{Hadoop-Common}} dependency also found in: LLAP, Serde, Storage, Shims,
> Shims Common, Shims Scheduler)
> {code}
> [INFO]
>
> [INFO] Building Hive Ant Utilities 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-ant ---
> [INFO] | +- commons-collections:commons-collections:jar:3.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO]
>
> [INFO] Building Hive Accumulo Handler 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.accumulo:accumulo-core:jar:1.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12469) Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address vulnerability
[
https://issues.apache.org/jira/browse/HIVE-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-12469:
Component/s: (was: Hive)
Build Infrastructure
> Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address
> vulnerability
> -
>
> Key: HIVE-12469
> URL: https://issues.apache.org/jira/browse/HIVE-12469
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
>Reporter: Reuben Kuhnert
>Assignee: Reuben Kuhnert
>Priority: Blocker
> Attachments: HIVE-12469.patch
>
>
> Currently the commons-collections (3.2.1) library allows for invocation of
> arbitrary code through {{InvokerTransformer}}, need to bump the version of
> commons-collections from 3.2.1 to 3.2.2 to resolve this issue.
> Results of {{mvn dependency:tree}}:
> {code}
> [INFO]
>
> [INFO] Building Hive HPL/SQL 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-hplsql ---
> [INFO] org.apache.hive:hive-hplsql:jar:2.0.0-SNAPSHOT
> [INFO] +- com.google.guava:guava:jar:14.0.1:compile
> [INFO] +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Packaging 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.hive:hive-hbase-handler:jar:2.0.0-SNAPSHOT:compile
> [INFO] | +- org.apache.hbase:hbase-server:jar:1.1.1:compile
> [INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Common 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-common ---
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {{Hadoop-Common}} dependency also found in: LLAP, Serde, Storage, Shims,
> Shims Common, Shims Scheduler)
> {code}
> [INFO]
>
> [INFO] Building Hive Ant Utilities 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-ant ---
> [INFO] | +- commons-collections:commons-collections:jar:3.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO]
>
> [INFO] Building Hive Accumulo Handler 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.accumulo:accumulo-core:jar:1.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12406: Attachment: (was: HIVE-12406.2.patch) > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12406: Attachment: HIVE-12406.2.patch > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12413) Default mode for hive.mapred.mode should be strict
[ https://issues.apache.org/jira/browse/HIVE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12413: Target Version/s: 2.0.0 > Default mode for hive.mapred.mode should be strict > -- > > Key: HIVE-12413 > URL: https://issues.apache.org/jira/browse/HIVE-12413 > Project: Hive > Issue Type: Task > Components: Configuration >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12413.1.patch, HIVE-12413.2.patch, HIVE-12413.patch > > > Non-strict mode allows some questionable semantics and questionable > operations. Its better that user makes a conscious choice to enable such a > behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018599#comment-15018599 ] Szehon Ho commented on HIVE-12406: -- Thanks Aihua! Do you think LazySimpleSerDe also need the annotations (the method is on there) ? > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-12479: Attachment: vector_test.q.out vector_test.q > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz, vector_test.q, vector_test.q.out > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-12479: Attachment: (was: vector_test.q.out) > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-12479: Attachment: (was: vector_test.q) > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12471) Secure HS2 web UI with SSL
[ https://issues.apache.org/jira/browse/HIVE-12471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-12471: --- Summary: Secure HS2 web UI with SSL (was: Secure HS2 web UI with SSL and kerberos) > Secure HS2 web UI with SSL > -- > > Key: HIVE-12471 > URL: https://issues.apache.org/jira/browse/HIVE-12471 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12436) Default hive.metastore.schema.verification to true
[ https://issues.apache.org/jira/browse/HIVE-12436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12436: Target Version/s: 2.0.0 > Default hive.metastore.schema.verification to true > -- > > Key: HIVE-12436 > URL: https://issues.apache.org/jira/browse/HIVE-12436 > Project: Hive > Issue Type: Task > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12436.patch > > > It enforces metastore schema version consistency -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12482) When execution.engine=tez,set mapreduce.job.name does not work.
[ https://issues.apache.org/jira/browse/HIVE-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018617#comment-15018617 ] Ashutosh Chauhan commented on HIVE-12482: - HIVE-12357 should help > When execution.engine=tez,set mapreduce.job.name does not work. > --- > > Key: HIVE-12482 > URL: https://issues.apache.org/jira/browse/HIVE-12482 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 0.14.0, 1.0.0, 1.0.1, 1.2.1 >Reporter: Xiaowei Wang > Fix For: 0.14.1 > > Attachments: HIVE-12482.0.patch > > > When execution.engine=tez,set mapreduce.job.name does not work. > In Tez mode, the default job name is "Hive_"+Sessionid ,for example > HIVE-ce5784d0-320c-4fb9-8b0b-2d92539dfd9e .It is difficulty to distinguish > job when there are too much jobs . > A better way is to set the var of mapreduce.job.name .But set > mapreduce.job.name does not work! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12331) Remove hive.enforce.bucketing & hive.enforce.sorting configs
[ https://issues.apache.org/jira/browse/HIVE-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12331: Target Version/s: 2.0.0 > Remove hive.enforce.bucketing & hive.enforce.sorting configs > > > Key: HIVE-12331 > URL: https://issues.apache.org/jira/browse/HIVE-12331 > Project: Hive > Issue Type: Improvement > Components: Configuration >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12331.1.patch, HIVE-12331.patch > > > If table is created as bucketed and/or sorted and this config is set to > false, you will insert data in wrong buckets and/or sort order and then if > you use these tables subsequently in BMJ or SMBJ you will get wrong results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12355) Keep Obj Inspectors in Sync with RowSchema
[ https://issues.apache.org/jira/browse/HIVE-12355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-12355: -- Assignee: Ashutosh Chauhan (was: Laljo John Pullokkaran) Target Version/s: 2.0.0 > Keep Obj Inspectors in Sync with RowSchema > -- > > Key: HIVE-12355 > URL: https://issues.apache.org/jira/browse/HIVE-12355 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 1.0.0, 1.1.0, 1.2.1 >Reporter: Laljo John Pullokkaran >Assignee: Ashutosh Chauhan > > Currently Not all operators match their Output Obj inspectors to Row schema. > Many times OutputObjectInspectors may be more than needed. > This causes problems especially with union. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12355) Keep Obj Inspectors in Sync with RowSchema
[ https://issues.apache.org/jira/browse/HIVE-12355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018649#comment-15018649 ] Laljo John Pullokkaran commented on HIVE-12355: --- [~ashutoshc] I think if we get TS to match OutputObjInsp to RowSchema (similar to Select) that might be sufficient. > Keep Obj Inspectors in Sync with RowSchema > -- > > Key: HIVE-12355 > URL: https://issues.apache.org/jira/browse/HIVE-12355 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 1.0.0, 1.1.0, 1.2.1 >Reporter: Laljo John Pullokkaran >Assignee: Ashutosh Chauhan > > Currently Not all operators match their Output Obj inspectors to Row schema. > Many times OutputObjectInspectors may be more than needed. > This causes problems especially with union. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12436) Default hive.metastore.schema.verification to true
[ https://issues.apache.org/jira/browse/HIVE-12436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018648#comment-15018648 ] Sushanth Sowmyan commented on HIVE-12436: - +1. :) > Default hive.metastore.schema.verification to true > -- > > Key: HIVE-12436 > URL: https://issues.apache.org/jira/browse/HIVE-12436 > Project: Hive > Issue Type: Task > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12436.patch > > > It enforces metastore schema version consistency -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7575) GetTables thrift call is very slow
[ https://issues.apache.org/jira/browse/HIVE-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-7575: Hadoop Flags: Incompatible change Release Note: This adds 5 additional columns to the ResultSet of GetTables. This is for compliance with the official JDBC API: See: http://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getTables(java.lang.String,%20java.lang.String,%20java.lang.String,%20java.lang.String[]) > GetTables thrift call is very slow > -- > > Key: HIVE-7575 > URL: https://issues.apache.org/jira/browse/HIVE-7575 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 0.12.0, 0.13.0 >Reporter: Ashu Pachauri >Assignee: Navis > Fix For: 2.0.0 > > Attachments: HIVE-7575.1.patch.txt, HIVE-7575.2.patch.txt, > HIVE-7575.3.patch.txt, HIVE-7575.4.patch.txt, HIVE-7575.5.patch.txt, > HIVE-7575.6.patch.txt, HIVE-7575.7.patch.txt > > > The GetTables thrift call takes a long time when the number of table is large. > With around 5000 tables, the call takes around 80 seconds compared to a "Show > Tables" query on the same HiveServer2 instance which takes 3-7 seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12469) Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address vulnerability
[
https://issues.apache.org/jira/browse/HIVE-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018680#comment-15018680
]
Reuben Kuhnert commented on HIVE-12469:
---
So that looks good to me for the most part, I guess my only question is this:
{code}
+3.2.1
1.9
1.1
3.0.1
@@ -303,7 +304,13 @@
commons-codec
${commons-codec.version}
-
+
+commons-collections
+commons-collections
+${commons-collections.version}
+provided
+
{code}
I would assume that at runtime this would still add
{{commons-collections-3.2.1}} to the runtime classpath (even if we do expect it
to be provided by the end user), which might re-introduce the issue. Feel free
to correct me if I'm wrong though.
> Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address
> vulnerability
> -
>
> Key: HIVE-12469
> URL: https://issues.apache.org/jira/browse/HIVE-12469
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
>Reporter: Reuben Kuhnert
>Assignee: Reuben Kuhnert
>Priority: Blocker
> Attachments: HIVE-12469.patch
>
>
> Currently the commons-collections (3.2.1) library allows for invocation of
> arbitrary code through {{InvokerTransformer}}, need to bump the version of
> commons-collections from 3.2.1 to 3.2.2 to resolve this issue.
> Results of {{mvn dependency:tree}}:
> {code}
> [INFO]
>
> [INFO] Building Hive HPL/SQL 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-hplsql ---
> [INFO] org.apache.hive:hive-hplsql:jar:2.0.0-SNAPSHOT
> [INFO] +- com.google.guava:guava:jar:14.0.1:compile
> [INFO] +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Packaging 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.hive:hive-hbase-handler:jar:2.0.0-SNAPSHOT:compile
> [INFO] | +- org.apache.hbase:hbase-server:jar:1.1.1:compile
> [INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Common 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-common ---
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {{Hadoop-Common}} dependency also found in: LLAP, Serde, Storage, Shims,
> Shims Common, Shims Scheduler)
> {code}
> [INFO]
>
> [INFO] Building Hive Ant Utilities 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-ant ---
> [INFO] | +- commons-collections:commons-collections:jar:3.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO]
>
> [INFO] Building Hive Accumulo Handler 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.accumulo:accumulo-core:jar:1.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12406: Attachment: (was: HIVE-12406.2.patch) > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12406: Attachment: HIVE-12406.2.patch > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[ https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018694#comment-15018694 ] Szehon Ho commented on HIVE-12406: -- +1 thanks! > HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface > > > Key: HIVE-12406 > URL: https://issues.apache.org/jira/browse/HIVE-12406 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Lenni Kuff >Assignee: Aihua Xu >Priority: Blocker > Attachments: HIVE-12406.2.patch, HIVE-12406.patch > > > In the process of fixing HIVE-9500, an incompatibility was introduced that > will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the > nested class SerDeParamaters was removed and the method > LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a > standalone class LazySerDeParameters. > Since this has already been released, I don't think we should revert the > change since that would mean breaking compatibility again. Instead, the best > approach would be to support both interfaces, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12319) Remove HadoopShims::getHadoopConfNames()
[ https://issues.apache.org/jira/browse/HIVE-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018700#comment-15018700 ] Aleksei Statkevich commented on HIVE-12319: --- Filed a jira HIVE-12487 and suggested a fix there. > Remove HadoopShims::getHadoopConfNames() > > > Key: HIVE-12319 > URL: https://issues.apache.org/jira/browse/HIVE-12319 > Project: Hive > Issue Type: Improvement > Components: Shims >Affects Versions: 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Aleksei Statkevich > Fix For: 2.0.0 > > Attachments: HIVE-12319.1.patch, HIVE-12319.patch > > > It was introduced in HIVE-6159 It has served its purpose now that we support > only Hadoop 2.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12487) Fix broken MiniLlap tests
[
https://issues.apache.org/jira/browse/HIVE-12487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksei Statkevich updated HIVE-12487:
--
Attachment: HIVE-12487.patch
I didn't find anything specific to HIVE-12319 in test logs. The only issue I
see when running locally is that aop library is missing which results in TEZ
execution being stuck. Which could potentially lead to result filen not being
produced.
After the change I'm able to successfully run one of llap itests:
{code}
mvn test -Dtest=TestMiniLlapCliDriver -pl qtest -Dqfile=bucket_map_join_tez1.q
{code}
> Fix broken MiniLlap tests
> -
>
> Key: HIVE-12487
> URL: https://issues.apache.org/jira/browse/HIVE-12487
> Project: Hive
> Issue Type: Bug
>Reporter: Aleksei Statkevich
>Assignee: Aleksei Statkevich
>Priority: Critical
> Attachments: HIVE-12487.patch
>
>
> Currently MiniLlap tests fail with the following error:
> {code}
> TestMiniLlapCliDriver - did not produce a TEST-*.xml file
> {code}
> Supposedly, it started happening after HIVE-12319.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11684) Implement limit pushdown through outer join in CBO
[ https://issues.apache.org/jira/browse/HIVE-11684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018710#comment-15018710 ] Laljo John Pullokkaran commented on HIVE-11684: --- +1 > Implement limit pushdown through outer join in CBO > -- > > Key: HIVE-11684 > URL: https://issues.apache.org/jira/browse/HIVE-11684 > Project: Hive > Issue Type: New Feature > Components: CBO >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-11684.01.patch, HIVE-11684.02.patch, > HIVE-11684.03.patch, HIVE-11684.04.patch, HIVE-11684.05.patch, > HIVE-11684.07.patch, HIVE-11684.08.patch, HIVE-11684.09.patch, > HIVE-11684.10.patch, HIVE-11684.11.patch, HIVE-11684.12.patch, > HIVE-11684.12.patch, HIVE-11684.14.patch, HIVE-11684.15.patch, > HIVE-11684.16.patch, HIVE-11684.17.patch, HIVE-11684.18.patch, > HIVE-11684.19.patch, HIVE-11684.20.patch, HIVE-11684.21.patch, > HIVE-11684.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018714#comment-15018714 ] Matt McCline commented on HIVE-12479: - VectorMapJoinInnerMultiKeyOperator closeOp 10 batches processed and 53 rows forwarded > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12456) QueryId can't be stored in the configuration of the SessionState since multiple queries can run in a single session
[ https://issues.apache.org/jira/browse/HIVE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-12456: Attachment: HIVE-12456.2.patch > QueryId can't be stored in the configuration of the SessionState since > multiple queries can run in a single session > --- > > Key: HIVE-12456 > URL: https://issues.apache.org/jira/browse/HIVE-12456 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-12456.2.patch, HIVE-12456.patch > > > Follow up on HIVE-11488 which stores the queryId in the sessionState conf. If > multiple queries run at the same time, then the logging will get wrong > queryId from the sessionState. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12483) Fix precommit Spark test branch
[
https://issues.apache.org/jira/browse/HIVE-12483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018746#comment-15018746
]
Hive QA commented on HIVE-12483:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12773549/HIVE-12483.1-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 9786 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1008/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1008/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-1008/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12773549 - PreCommit-HIVE-SPARK-Build
> Fix precommit Spark test branch
> ---
>
> Key: HIVE-12483
> URL: https://issues.apache.org/jira/browse/HIVE-12483
> Project: Hive
> Issue Type: Task
>Reporter: Sergio Peña
>Assignee: Sergio Peña
> Attachments: HIVE-12483.1-spark.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12456) QueryId can't be stored in the configuration of the SessionState since multiple queries can run in a single session
[ https://issues.apache.org/jira/browse/HIVE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018748#comment-15018748 ] Aihua Xu commented on HIVE-12456: - Uploaded a new patch to address comments. > QueryId can't be stored in the configuration of the SessionState since > multiple queries can run in a single session > --- > > Key: HIVE-12456 > URL: https://issues.apache.org/jira/browse/HIVE-12456 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-12456.2.patch, HIVE-12456.patch > > > Follow up on HIVE-11488 which stores the queryId in the sessionState conf. If > multiple queries run at the same time, then the logging will get wrong > queryId from the sessionState. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12456) QueryId can't be stored in the configuration of the SessionState since multiple queries can run in a single session
[ https://issues.apache.org/jira/browse/HIVE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018782#comment-15018782 ] Mohit Sabharwal commented on HIVE-12456: thanks, LGTM. +1(non-binding) > QueryId can't be stored in the configuration of the SessionState since > multiple queries can run in a single session > --- > > Key: HIVE-12456 > URL: https://issues.apache.org/jira/browse/HIVE-12456 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-12456.2.patch, HIVE-12456.patch > > > Follow up on HIVE-11488 which stores the queryId in the sessionState conf. If > multiple queries run at the same time, then the logging will get wrong > queryId from the sessionState. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12160) Hbase table query execution fails in secured cluster when hive.exec.mode.local.auto is set to true
[
https://issues.apache.org/jira/browse/HIVE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018775#comment-15018775
]
Hive QA commented on HIVE-12160:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12773011/HIVE-12160.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9831 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.jdbc.miniHS2.TestHs2Metrics.testMetrics
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6083/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6083/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6083/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12773011 - PreCommit-HIVE-TRUNK-Build
> Hbase table query execution fails in secured cluster when
> hive.exec.mode.local.auto is set to true
> --
>
> Key: HIVE-12160
> URL: https://issues.apache.org/jira/browse/HIVE-12160
> Project: Hive
> Issue Type: Bug
> Components: Security
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-12160.patch, HIVE-12160_trace.txt
>
>
> In a secured cluster with kerberos, a simple query like {{select count(*)
> from hbase_table;}} will fail with the following exception when
> hive.exec.mode.local.auto is set to true.
> {noformat}
> Error: Error while processing statement: FAILED: Execution Error, return code
> 134 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=134)
> {noformat}
> There is another scenario which may be caused by the same reason.
> Set hive.auto.convert.join to true, the join query {{select * from hbase_t1
> join hbase_t2 on hbase_t1.id = hbase_t2.id;}} also fails with the following
> exception:
> {noformat}
> Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask (state=08S01,code=2)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12413) Default mode for hive.mapred.mode should be strict
[
https://issues.apache.org/jira/browse/HIVE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018784#comment-15018784
]
Hive QA commented on HIVE-12413:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12773042/HIVE-12413.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6084/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6084/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6084/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-6084/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 97cb0c6 HIVE-12450: OrcFileMergeOperator does not use correct
compression buffer size (Prasanth Jayachandran reviewed by Sergey Shelukhin)
+ git clean -f -d
Removing ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java.orig
+ git checkout master
Already on 'master'
+ git reset --hard origin/master
HEAD is now at 97cb0c6 HIVE-12450: OrcFileMergeOperator does not use correct
compression buffer size (Prasanth Jayachandran reviewed by Sergey Shelukhin)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12773042 - PreCommit-HIVE-TRUNK-Build
> Default mode for hive.mapred.mode should be strict
> --
>
> Key: HIVE-12413
> URL: https://issues.apache.org/jira/browse/HIVE-12413
> Project: Hive
> Issue Type: Task
> Components: Configuration
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12413.1.patch, HIVE-12413.2.patch, HIVE-12413.patch
>
>
> Non-strict mode allows some questionable semantics and questionable
> operations. Its better that user makes a conscious choice to enable such a
> behavior.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12465) Hive might produce wrong results when (outer) joins are merged
[
https://issues.apache.org/jira/browse/HIVE-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-12465:
---
Attachment: HIVE-12465.patch
> Hive might produce wrong results when (outer) joins are merged
> --
>
> Key: HIVE-12465
> URL: https://issues.apache.org/jira/browse/HIVE-12465
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Attachments: HIVE-12465.patch
>
>
> Consider the following query:
> {noformat}
> select * from
> (select * from tab where tab.key = 0)a
> full outer join
> (select * from tab_part where tab_part.key = 98)b
> join
> tab_part c
> on a.key = b.key and b.key = c.key;
> {noformat}
> Hive should execute the full outer join operation (without ON clause) and
> then the join operation (ON a.key = b.key and b.key = c.key). Instead, it
> merges both joins, generating the following plan:
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: tab
> filterExpr: (key = 0) (type: boolean)
> Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (key = 0) (type: boolean)
> Statistics: Num rows: 121 Data size: 11374 Basic stats:
> COMPLETE Column stats: NONE
> Select Operator
> expressions: 0 (type: int), value (type: string), ds (type:
> string)
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 121 Data size: 11374 Basic stats:
> COMPLETE Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (type: int)
> sort order: +
> Map-reduce partition columns: _col0 (type: int)
> Statistics: Num rows: 121 Data size: 11374 Basic stats:
> COMPLETE Column stats: NONE
> value expressions: _col1 (type: string), _col2 (type:
> string)
> TableScan
> alias: tab_part
> filterExpr: (key = 98) (type: boolean)
> Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (key = 98) (type: boolean)
> Statistics: Num rows: 250 Data size: 23500 Basic stats:
> COMPLETE Column stats: NONE
> Select Operator
> expressions: 98 (type: int), value (type: string), ds (type:
> string)
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 250 Data size: 23500 Basic stats:
> COMPLETE Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (type: int)
> sort order: +
> Map-reduce partition columns: _col0 (type: int)
> Statistics: Num rows: 250 Data size: 23500 Basic stats:
> COMPLETE Column stats: NONE
> value expressions: _col1 (type: string), _col2 (type:
> string)
> TableScan
> alias: c
> Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: key (type: int)
> sort order: +
> Map-reduce partition columns: key (type: int)
> Statistics: Num rows: 500 Data size: 47000 Basic stats:
> COMPLETE Column stats: NONE
> value expressions: value (type: string), ds (type: string)
> Reduce Operator Tree:
> Join Operator
> condition map:
>Outer Join 0 to 1
>Inner Join 1 to 2
> keys:
> 0 _col0 (type: int)
> 1 _col0 (type: int)
> 2 key (type: int)
> outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6,
> _col7, _col8
> Statistics: Num rows: 1100 Data size: 103400 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 1100 Data size: 103400 Basic stats:
> COMPLETE Column stats: NONE
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
>
[jira] [Updated] (HIVE-12486) Using temporary functions fail when using hive whitelist
[
https://issues.apache.org/jira/browse/HIVE-12486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sravya Tirukkovalur updated HIVE-12486:
---
Description:
CREATE TEMPORARY FUNCTION printf_test AS
'org.apache.hadoop.hive.ql.udf.generic.GenericUDFPrintf'
SELECT printf_test('%d', under_col) FROM tab1;
The above select fails with
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement:
FAILED: SemanticException UDF printf_test is not allowed
> Using temporary functions fail when using hive whitelist
>
>
> Key: HIVE-12486
> URL: https://issues.apache.org/jira/browse/HIVE-12486
> Project: Hive
> Issue Type: Bug
>Reporter: Sravya Tirukkovalur
>
> CREATE TEMPORARY FUNCTION printf_test AS
> 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFPrintf'
> SELECT printf_test('%d', under_col) FROM tab1;
> The above select fails with
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: SemanticException UDF printf_test is not allowed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12489) Analyze for partition fails if partition value has special characters
[ https://issues.apache.org/jira/browse/HIVE-12489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Friedrich updated HIVE-12489: Attachment: HIVE-12489.patch > Analyze for partition fails if partition value has special characters > - > > Key: HIVE-12489 > URL: https://issues.apache.org/jira/browse/HIVE-12489 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Thomas Friedrich >Assignee: Thomas Friedrich >Priority: Minor > Attachments: HIVE-12489.patch > > > When analyzing a partition that has a special characters in the value, the > analyze command fails with an exception. > Example: > hive> create table testtable (a int) partitioned by (b string); > hive> insert into table testtable partition (b="p\"1") values (1); > hive> ANALYZE TABLE testtable PARTITION(b="p\"1") COMPUTE STATISTICS for > columns a; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12045) ClassNotFound for GenericUDF in "select distinct..." query (Hive on Spark)
[
https://issues.apache.org/jira/browse/HIVE-12045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018886#comment-15018886
]
Xuefu Zhang commented on HIVE-12045:
+1 to latest patch.
> ClassNotFound for GenericUDF in "select distinct..." query (Hive on Spark)
> --
>
> Key: HIVE-12045
> URL: https://issues.apache.org/jira/browse/HIVE-12045
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Environment: Cloudera QuickStart VM - CDH5.4.2
> beeline
>Reporter: Zsolt Tóth
>Assignee: Rui Li
> Attachments: HIVE-12045.1-spark.patch, HIVE-12045.2-spark.patch,
> HIVE-12045.2-spark.patch, HIVE-12045.3-spark.patch, HIVE-12045.4-spark.patch,
> HIVE-12045.patch, example.jar, genUDF.patch, hive.log.gz
>
>
> If I execute the following query in beeline, I get ClassNotFoundException for
> the UDF class.
> {code}
> drop function myGenericUdf;
> create function myGenericUdf as 'org.example.myGenericUdf' using jar
> 'hdfs:///tmp/myudf.jar';
> select distinct myGenericUdf(1,2,1) from mytable;
> {code}
> In my example, myGenericUdf just looks for the 1st argument's value in the
> others and returns the index. I don't think this is related to the actual
> GenericUDF function.
> Note that:
> "select myGenericUdf(1,2,1) from mytable;" succeeds
> If I use the non-generic implementation of the same UDF, the select distinct
> call succeeds.
> StackTrace:
> {code}
> 15/10/06 05:20:25 ERROR exec.Utilities: Failed to load plan:
> hdfs://quickstart.cloudera:8020/tmp/hive/hive/f9de3f09-c12d-4528-9ee6-1f12932a14ae/hive_2015-10-06_05-20-07_438_6519207588897968406-20/-mr-10003/27cd7226-3e22-46f4-bddd-fb8fd4aa4b8d/map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find
> class: org.example.myGenericUDF
> Serialization trace:
> genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> colExprMap (org.apache.hadoop.hive.ql.exec.GroupByOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find
> class: org.example.myGenericUDF
> Serialization trace:
> genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> colExprMap (org.apache.hadoop.hive.ql.exec.GroupByOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:99)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at
> org.apach
[jira] [Commented] (HIVE-12489) Analyze for partition fails if partition value has special characters
[ https://issues.apache.org/jira/browse/HIVE-12489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018898#comment-15018898 ] Ashutosh Chauhan commented on HIVE-12489: - +1 pending test > Analyze for partition fails if partition value has special characters > - > > Key: HIVE-12489 > URL: https://issues.apache.org/jira/browse/HIVE-12489 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Thomas Friedrich >Assignee: Thomas Friedrich >Priority: Minor > Attachments: HIVE-12489.patch > > > When analyzing a partition that has a special characters in the value, the > analyze command fails with an exception. > Example: > hive> create table testtable (a int) partitioned by (b string); > hive> insert into table testtable partition (b="p\"1") values (1); > hive> ANALYZE TABLE testtable PARTITION(b="p\"1") COMPUTE STATISTICS for > columns a; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12367) Lock/unlock database should add current database to inputs and outputs of authz hook
[ https://issues.apache.org/jira/browse/HIVE-12367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018911#comment-15018911 ] Alan Gates commented on HIVE-12367: --- I agree that DDL_NO_LOCK is the right annotation, even on the lock side. I think a comment on why it's right in the code would be helpful, since when I first looked at it it confused me. As for whether you do the tables in this ticket or another is up to you, I think either is fine. > Lock/unlock database should add current database to inputs and outputs of > authz hook > > > Key: HIVE-12367 > URL: https://issues.apache.org/jira/browse/HIVE-12367 > Project: Hive > Issue Type: Bug > Components: Authorization >Affects Versions: 1.2.1 >Reporter: Dapeng Sun >Assignee: Dapeng Sun > Attachments: HIVE-12367.001.patch, HIVE-12367.002.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12469) Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address vulnerability
[
https://issues.apache.org/jira/browse/HIVE-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018910#comment-15018910
]
Ashutosh Chauhan commented on HIVE-12469:
-
yeah.. I dont know if there is something better we can do here short of
updating all our immediate dependencies to their respective versions which
don't use 3.2.1 version. This patch alteast makes sure that Hive is not
bringing in offending version on run time classpath.
> Bump Commons-Collections dependency from 3.2.1 to 3.2.2. to address
> vulnerability
> -
>
> Key: HIVE-12469
> URL: https://issues.apache.org/jira/browse/HIVE-12469
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
>Reporter: Reuben Kuhnert
>Assignee: Reuben Kuhnert
>Priority: Blocker
> Attachments: HIVE-12469.patch
>
>
> Currently the commons-collections (3.2.1) library allows for invocation of
> arbitrary code through {{InvokerTransformer}}, need to bump the version of
> commons-collections from 3.2.1 to 3.2.2 to resolve this issue.
> Results of {{mvn dependency:tree}}:
> {code}
> [INFO]
>
> [INFO] Building Hive HPL/SQL 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-hplsql ---
> [INFO] org.apache.hive:hive-hplsql:jar:2.0.0-SNAPSHOT
> [INFO] +- com.google.guava:guava:jar:14.0.1:compile
> [INFO] +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Packaging 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.hive:hive-hbase-handler:jar:2.0.0-SNAPSHOT:compile
> [INFO] | +- org.apache.hbase:hbase-server:jar:1.1.1:compile
> [INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO] Building Hive Common 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-common ---
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
> {{Hadoop-Common}} dependency also found in: LLAP, Serde, Storage, Shims,
> Shims Common, Shims Scheduler)
> {code}
> [INFO]
>
> [INFO] Building Hive Ant Utilities 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-ant ---
> [INFO] | +- commons-collections:commons-collections:jar:3.1:compile
> {code}
> {code}
> [INFO]
>
> [INFO]
>
> [INFO] Building Hive Accumulo Handler 2.0.0-SNAPSHOT
> [INFO]
>
> [INFO] +- org.apache.accumulo:accumulo-core:jar:1.6.0:compile
> [INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12045) ClassNotFoundException for GenericUDF [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-12045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-12045:
---
Summary: ClassNotFoundException for GenericUDF [Spark Branch] (was:
ClassNotFound for GenericUDF in "select distinct..." query (Hive on Spark))
> ClassNotFoundException for GenericUDF [Spark Branch]
>
>
> Key: HIVE-12045
> URL: https://issues.apache.org/jira/browse/HIVE-12045
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Environment: Cloudera QuickStart VM - CDH5.4.2
> beeline
>Reporter: Zsolt Tóth
>Assignee: Rui Li
> Attachments: HIVE-12045.1-spark.patch, HIVE-12045.2-spark.patch,
> HIVE-12045.2-spark.patch, HIVE-12045.3-spark.patch, HIVE-12045.4-spark.patch,
> HIVE-12045.patch, example.jar, genUDF.patch, hive.log.gz
>
>
> If I execute the following query in beeline, I get ClassNotFoundException for
> the UDF class.
> {code}
> drop function myGenericUdf;
> create function myGenericUdf as 'org.example.myGenericUdf' using jar
> 'hdfs:///tmp/myudf.jar';
> select distinct myGenericUdf(1,2,1) from mytable;
> {code}
> In my example, myGenericUdf just looks for the 1st argument's value in the
> others and returns the index. I don't think this is related to the actual
> GenericUDF function.
> Note that:
> "select myGenericUdf(1,2,1) from mytable;" succeeds
> If I use the non-generic implementation of the same UDF, the select distinct
> call succeeds.
> StackTrace:
> {code}
> 15/10/06 05:20:25 ERROR exec.Utilities: Failed to load plan:
> hdfs://quickstart.cloudera:8020/tmp/hive/hive/f9de3f09-c12d-4528-9ee6-1f12932a14ae/hive_2015-10-06_05-20-07_438_6519207588897968406-20/-mr-10003/27cd7226-3e22-46f4-bddd-fb8fd4aa4b8d/map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find
> class: org.example.myGenericUDF
> Serialization trace:
> genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> colExprMap (org.apache.hadoop.hive.ql.exec.GroupByOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find
> class: org.example.myGenericUDF
> Serialization trace:
> genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> colExprMap (org.apache.hadoop.hive.ql.exec.GroupByOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:99)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:
[jira] [Commented] (HIVE-12319) Remove HadoopShims::getHadoopConfNames()
[ https://issues.apache.org/jira/browse/HIVE-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018931#comment-15018931 ] Ashutosh Chauhan commented on HIVE-12319: - [~sershe] I am not able to repro this on fresh master. Further http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6083/#showFailuresLink which happened after build # 6078 didnt have llap test failures. I am not sure how you came to conclusion that this patch has broken llap tests? > Remove HadoopShims::getHadoopConfNames() > > > Key: HIVE-12319 > URL: https://issues.apache.org/jira/browse/HIVE-12319 > Project: Hive > Issue Type: Improvement > Components: Shims >Affects Versions: 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Aleksei Statkevich > Fix For: 2.0.0 > > Attachments: HIVE-12319.1.patch, HIVE-12319.patch > > > It was introduced in HIVE-6159 It has served its purpose now that we support > only Hadoop 2.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11356) SMB join on tez fails when one of the tables is empty
[
https://issues.apache.org/jira/browse/HIVE-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikram Dixit K updated HIVE-11356:
--
Fix Version/s: 1.2.2
1.3.0
> SMB join on tez fails when one of the tables is empty
> -
>
> Key: HIVE-11356
> URL: https://issues.apache.org/jira/browse/HIVE-11356
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Fix For: 1.3.0, 2.0.0, 1.2.2
>
> Attachments: HIVE-11356.1.patch, HIVE-11356.3.patch,
> HIVE-11356.4.patch, HIVE-11356.5.patch, HIVE-11356.6.patch
>
>
> {code}
> :java.lang.IllegalStateException: Unexpected event. All physical sources
> already initialized
> at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at
> org.apache.tez.mapreduce.input.MultiMRInput.handleEvents(MultiMRInput.java:142)
>
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:610)
>
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$1100(LogicalIOProcessorRuntimeTask.java:90)
>
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.run(LogicalIOProcessorRuntimeTask.java:673)
>
> at java.lang.Thread.run(Thread.java:745)
> ]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex
> vertex_1437168420060_17787_1_01 [Map 4] killed/failed due to:null]
> Vertex killed, vertexName=Reducer 5,
> vertexId=vertex_1437168420060_17787_1_02, diagnostics=[Vertex received Kill
> while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0,
> Vertex vertex_1437168420060_17787_1_02 [Reducer 5] killed/failed due to:null]
> DAG failed due to vertex failure. failedVertices:1 killedVertices:1
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask
> HQL-FAILED
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018947#comment-15018947 ] Matt McCline commented on HIVE-12479: - Turning off the Native Vector ReduceSink does not eliminate the problem. Current guess: old pass-thru VectorMapJoinOperator passes a separate output batch with an identity (i.e. [0, 1, 2, 3, 4]) projection array. VectorMapJoinMultiKeyOperator uses the input BigTable batch for output and has a non-identity projection (i.e. [0, 3, 4, 0, 1] -- don't worry about the repeated 0). How does the VectorFilterOperator handle projection? > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12479) Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the middle
[ https://issues.apache.org/jira/browse/HIVE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018952#comment-15018952 ] Matt McCline commented on HIVE-12479: - It doesn't appear that the VectorFilterOperator handles vectorized row batch projection at all. > Vectorization: vectorized MultiKeyJoins with 2 integers with a filter in the > middle > --- > > Key: HIVE-12479 > URL: https://issues.apache.org/jira/browse/HIVE-12479 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 1.3.0, 2.0.0 >Reporter: Gopal V >Assignee: Matt McCline > Attachments: HIVE-12479.tar.gz > > > The row-counts expected with and without vectorization differ. > The attached small-scale repro case produces 5 rows with vectorized multi-key > joins and 53 rows without the vectorized join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11180) Enable native vectorized map join for spark [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-11180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018966#comment-15018966 ] Vikram Dixit K commented on HIVE-11180: --- The backport of this patch to the branch-1.2 has broken the build of branch 1.2: commit id: fa6f5cf674991d2d7146da537750bb2bf5b6b339 This commit did not merge HiveConf.java properly and removed pre-existing configuration: HIVEMAPJOINOPTIMIZEDTABLEPROBEPERCENT > Enable native vectorized map join for spark [Spark Branch] > -- > > Key: HIVE-11180 > URL: https://issues.apache.org/jira/browse/HIVE-11180 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Labels: TODOC-SPARK, TODOC1.3 > Fix For: spark-branch, 1.3.0, 2.0.0 > > Attachments: HIVE-11180.1-spark.patch, HIVE-11180.2-spark.patch > > > The improvement was introduced in HIVE-9824. Let's use this task to track how > we can enable that for spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-11180) Enable native vectorized map join for spark [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-11180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018966#comment-15018966 ] Vikram Dixit K edited comment on HIVE-11180 at 11/20/15 10:48 PM: -- The backport of this patch has broken the build of branch 1.2 and branch-1: commit id: fa6f5cf674991d2d7146da537750bb2bf5b6b339 This commit did not merge HiveConf.java properly and removed pre-existing configuration: HIVEMAPJOINOPTIMIZEDTABLEPROBEPERCENT was (Author: vikram.dixit): The backport of this patch to the branch-1.2 has broken the build of branch 1.2: commit id: fa6f5cf674991d2d7146da537750bb2bf5b6b339 This commit did not merge HiveConf.java properly and removed pre-existing configuration: HIVEMAPJOINOPTIMIZEDTABLEPROBEPERCENT > Enable native vectorized map join for spark [Spark Branch] > -- > > Key: HIVE-11180 > URL: https://issues.apache.org/jira/browse/HIVE-11180 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Labels: TODOC-SPARK, TODOC1.3 > Fix For: spark-branch, 1.3.0, 2.0.0 > > Attachments: HIVE-11180.1-spark.patch, HIVE-11180.2-spark.patch > > > The improvement was introduced in HIVE-9824. Let's use this task to track how > we can enable that for spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12486) Using temporary functions fail when using hive whitelist
[
https://issues.apache.org/jira/browse/HIVE-12486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018976#comment-15018976
]
Sravya Tirukkovalur commented on HIVE-12486:
Seems like we should instead check if the udf corresponding to the class has
been whitelisted instead of the alias.
> Using temporary functions fail when using hive whitelist
>
>
> Key: HIVE-12486
> URL: https://issues.apache.org/jira/browse/HIVE-12486
> Project: Hive
> Issue Type: Bug
>Reporter: Sravya Tirukkovalur
>
> CREATE TEMPORARY FUNCTION printf_test AS
> 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFPrintf'
> SELECT printf_test('%d', under_col) FROM tab1;
> The above select fails with
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: SemanticException UDF printf_test is not allowed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-11180) Enable native vectorized map join for spark [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-11180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018966#comment-15018966 ] Vikram Dixit K edited comment on HIVE-11180 at 11/20/15 10:49 PM: -- The backport of this patch has broken the build of branch-1: commit id: fa6f5cf674991d2d7146da537750bb2bf5b6b339 This commit did not merge HiveConf.java properly and removed pre-existing configuration: HIVEMAPJOINOPTIMIZEDTABLEPROBEPERCENT was (Author: vikram.dixit): The backport of this patch has broken the build of branch 1.2 and branch-1: commit id: fa6f5cf674991d2d7146da537750bb2bf5b6b339 This commit did not merge HiveConf.java properly and removed pre-existing configuration: HIVEMAPJOINOPTIMIZEDTABLEPROBEPERCENT > Enable native vectorized map join for spark [Spark Branch] > -- > > Key: HIVE-11180 > URL: https://issues.apache.org/jira/browse/HIVE-11180 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Labels: TODOC-SPARK, TODOC1.3 > Fix For: spark-branch, 1.3.0, 2.0.0 > > Attachments: HIVE-11180.1-spark.patch, HIVE-11180.2-spark.patch > > > The improvement was introduced in HIVE-9824. Let's use this task to track how > we can enable that for spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12319) Remove HadoopShims::getHadoopConfNames()
[ https://issues.apache.org/jira/browse/HIVE-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018992#comment-15018992 ] Sergey Shelukhin commented on HIVE-12319: - The tests didn't run at all, this for some reason doesn't show in failures: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6083/testReport/org.apache.hadoop.hive.cli/ The way I came to the conclusion is that the test started failing like that after this build, and this is the only JIRA in that build where HiveQA before commit failed like that. > Remove HadoopShims::getHadoopConfNames() > > > Key: HIVE-12319 > URL: https://issues.apache.org/jira/browse/HIVE-12319 > Project: Hive > Issue Type: Improvement > Components: Shims >Affects Versions: 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Aleksei Statkevich > Fix For: 2.0.0 > > Attachments: HIVE-12319.1.patch, HIVE-12319.patch > > > It was introduced in HIVE-6159 It has served its purpose now that we support > only Hadoop 2.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12487) Fix broken MiniLlap tests
[
https://issues.apache.org/jira/browse/HIVE-12487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018993#comment-15018993
]
Sergey Shelukhin commented on HIVE-12487:
-
If you are saying you cannot repro the issue might be somehow specific to
HiveQA setup... I'll take a look a bit later
> Fix broken MiniLlap tests
> -
>
> Key: HIVE-12487
> URL: https://issues.apache.org/jira/browse/HIVE-12487
> Project: Hive
> Issue Type: Bug
>Reporter: Aleksei Statkevich
>Assignee: Aleksei Statkevich
>Priority: Critical
> Attachments: HIVE-12487.patch
>
>
> Currently MiniLlap tests fail with the following error:
> {code}
> TestMiniLlapCliDriver - did not produce a TEST-*.xml file
> {code}
> Supposedly, it started happening after HIVE-12319.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12487) Fix broken MiniLlap tests
[
https://issues.apache.org/jira/browse/HIVE-12487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019004#comment-15019004
]
Aleksei Statkevich commented on HIVE-12487:
---
I was able to successfully run locally a large portion of qfile tests in Llap,
so I think that test failure is unlikely to be caused by a code change.
Another potential reason for tests failure with no TEST-*.xml file is that
"orc_ppd_basic.q" test took almost 2 hours on my machine, so it could easily
have been killed by a timeout. This is a more likely reason which has some
confirmation in log. Splitting or temporary disabling this test will help.
In any case, so far I haven't seen any relationship between Llap test failure
and a change in HIVE-12319.
> Fix broken MiniLlap tests
> -
>
> Key: HIVE-12487
> URL: https://issues.apache.org/jira/browse/HIVE-12487
> Project: Hive
> Issue Type: Bug
>Reporter: Aleksei Statkevich
>Assignee: Aleksei Statkevich
>Priority: Critical
> Attachments: HIVE-12487.patch
>
>
> Currently MiniLlap tests fail with the following error:
> {code}
> TestMiniLlapCliDriver - did not produce a TEST-*.xml file
> {code}
> Supposedly, it started happening after HIVE-12319.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12487) Fix broken MiniLlap tests
[
https://issues.apache.org/jira/browse/HIVE-12487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019011#comment-15019011
]
Sergey Shelukhin commented on HIVE-12487:
-
Oh, yeah, that's definitely a timeout then, the timeout on HiveQA is 1 hour.
The test runtime might have been increased by changes in HIVE-12319, because
these failures never happened before (since we merged to master).
Maybe some configuration has changed for some code... I will take a look a bit
when I have some cycles.
> Fix broken MiniLlap tests
> -
>
> Key: HIVE-12487
> URL: https://issues.apache.org/jira/browse/HIVE-12487
> Project: Hive
> Issue Type: Bug
>Reporter: Aleksei Statkevich
>Assignee: Aleksei Statkevich
>Priority: Critical
> Attachments: HIVE-12487.patch
>
>
> Currently MiniLlap tests fail with the following error:
> {code}
> TestMiniLlapCliDriver - did not produce a TEST-*.xml file
> {code}
> Supposedly, it started happening after HIVE-12319.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12472) Add test case for HIVE-10592
[
https://issues.apache.org/jira/browse/HIVE-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019062#comment-15019062
]
Ashutosh Chauhan commented on HIVE-12472:
-
+1
> Add test case for HIVE-10592
>
>
> Key: HIVE-12472
> URL: https://issues.apache.org/jira/browse/HIVE-12472
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-12472.patch
>
>
> HIVE-10592 has a fix for the following NPE issue (table should have all
> columns values as null for timestamp and date columns)
> {code:title=query}
> set hive.optimize.index.filter=true;
> select count(*) from orctable where timestamp_col is null;
> select count(*) from orctable where date_col is null;
> {code}
> {code:title=exception}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$TimestampStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:845)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getMin(RecordReaderImpl.java:308)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:332)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.(RecordReaderImpl.java:205)
> at
> org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.(OrcRawRecordMerger.java:183)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.(OrcRawRecordMerger.java:226)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:437)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1235)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1117)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:249)
> ... 26 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:1, Vertex vertex_1446768202865_0008_5_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-12472) Add test case for HIVE-10592
[
https://issues.apache.org/jira/browse/HIVE-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran resolved HIVE-12472.
--
Resolution: Fixed
Committed to branch-1 and master.
> Add test case for HIVE-10592
>
>
> Key: HIVE-12472
> URL: https://issues.apache.org/jira/browse/HIVE-12472
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-12472.patch
>
>
> HIVE-10592 has a fix for the following NPE issue (table should have all
> columns values as null for timestamp and date columns)
> {code:title=query}
> set hive.optimize.index.filter=true;
> select count(*) from orctable where timestamp_col is null;
> select count(*) from orctable where date_col is null;
> {code}
> {code:title=exception}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$TimestampStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:845)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getMin(RecordReaderImpl.java:308)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:332)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.(RecordReaderImpl.java:205)
> at
> org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.(OrcRawRecordMerger.java:183)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.(OrcRawRecordMerger.java:226)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:437)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1235)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1117)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:249)
> ... 26 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:1, Vertex vertex_1446768202865_0008_5_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10613) HCatSchemaUtils getHCatFieldSchema should include field comment
[ https://issues.apache.org/jira/browse/HIVE-10613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Friedrich updated HIVE-10613: Attachment: (was: HIVE-10613.patch) > HCatSchemaUtils getHCatFieldSchema should include field comment > --- > > Key: HIVE-10613 > URL: https://issues.apache.org/jira/browse/HIVE-10613 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 1.0.0 >Reporter: Thomas Friedrich >Assignee: Thomas Friedrich >Priority: Minor > > HCatSchemaUtils.getHCatFieldSchema converts a FieldSchema to a > HCatFieldSchema. Instead of initializing the comment property from the > FieldSchema object, the comment in the HCatFieldSchema is always set to null. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10613) HCatSchemaUtils getHCatFieldSchema should include field comment
[ https://issues.apache.org/jira/browse/HIVE-10613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Friedrich updated HIVE-10613: Attachment: HIVE-10613.patch > HCatSchemaUtils getHCatFieldSchema should include field comment > --- > > Key: HIVE-10613 > URL: https://issues.apache.org/jira/browse/HIVE-10613 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 1.0.0 >Reporter: Thomas Friedrich >Assignee: Thomas Friedrich >Priority: Minor > Attachments: HIVE-10613.patch > > > HCatSchemaUtils.getHCatFieldSchema converts a FieldSchema to a > HCatFieldSchema. Instead of initializing the comment property from the > FieldSchema object, the comment in the HCatFieldSchema is always set to null. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12406) HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
[
https://issues.apache.org/jira/browse/HIVE-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019155#comment-15019155
]
Hive QA commented on HIVE-12406:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12773587/HIVE-12406.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9832 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6085/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6085/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6085/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12773587 - PreCommit-HIVE-TRUNK-Build
> HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
>
>
> Key: HIVE-12406
> URL: https://issues.apache.org/jira/browse/HIVE-12406
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Affects Versions: 1.2.0
>Reporter: Lenni Kuff
>Assignee: Aihua Xu
>Priority: Blocker
> Attachments: HIVE-12406.2.patch, HIVE-12406.patch
>
>
> In the process of fixing HIVE-9500, an incompatibility was introduced that
> will break 3rd party code that relies on LazySimpleSerde. In HIVE-9500, the
> nested class SerDeParamaters was removed and the method
> LazySimpleSerDe.initSerdeParms was also removed. They were replaced by a
> standalone class LazySerDeParameters.
> Since this has already been released, I don't think we should revert the
> change since that would mean breaking compatibility again. Instead, the best
> approach would be to support both interfaces, if possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12444) Simple Fetch optimization on ACID table without base directory may throw exception
[
https://issues.apache.org/jira/browse/HIVE-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Zheng updated HIVE-12444:
-
Summary: Simple Fetch optimization on ACID table without base directory may
throw exception (was: Queries against ACID table without base directory may
throw exception)
> Simple Fetch optimization on ACID table without base directory may throw
> exception
> --
>
> Key: HIVE-12444
> URL: https://issues.apache.org/jira/browse/HIVE-12444
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.2.1
>Reporter: Wei Zheng
>Assignee: Wei Zheng
>
> Steps to reproduce:
> set hive.fetch.task.conversion=minimal;
> set hive.limit.optimize.enable=true;
> create table acidtest1(
> c_custkey int,
> c_name string,
> c_nationkey int,
> c_acctbal double)
> clustered by (c_nationkey) into 3 buckets
> stored as orc
> tblproperties("transactional"="true");
> insert into table acidtest1
> select c_custkey, c_name, c_nationkey, c_acctbal from tpch_text_10.customer;
> select cast (c_nationkey as string) from acidtest.acidtest1 limit 10;
> {code}
> DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1,
> vertexId=vertex_1447362491939_0020_1_00, diagnostics=[Vertex
> vertex_1447362491939_0020_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: acidtest1 initializer failed,
> vertex=vertex_1447362491939_0020_1_00 [Map 1], java.lang.RuntimeException:
> serious problem
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1035)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1062)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:308)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:410)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:155)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:246)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:240)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:227)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: delta_017_017 does not start with
> base_
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1012)
> ... 15 more
> Caused by: java.lang.IllegalArgumentException: delta_017_017 does not
> start with base_
> at org.apache.hadoop.hive.ql.io.AcidUtils.parseBase(AcidUtils.java:144)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.parseBaseBucketFilename(AcidUtils.java:172)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$FileGenerator.call(OrcInputFormat.java:667)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$FileGenerator.call(OrcInputFormat.java:625)
> ... 4 more
> ]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12482) When execution.engine=tez,set mapreduce.job.name does not work.
[ https://issues.apache.org/jira/browse/HIVE-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019183#comment-15019183 ] Xiaowei Wang commented on HIVE-12482: - Thanks very much ! I close this jira! > When execution.engine=tez,set mapreduce.job.name does not work. > --- > > Key: HIVE-12482 > URL: https://issues.apache.org/jira/browse/HIVE-12482 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 0.14.0, 1.0.0, 1.0.1, 1.2.1 >Reporter: Xiaowei Wang > Fix For: 0.14.1 > > Attachments: HIVE-12482.0.patch > > > When execution.engine=tez,set mapreduce.job.name does not work. > In Tez mode, the default job name is "Hive_"+Sessionid ,for example > HIVE-ce5784d0-320c-4fb9-8b0b-2d92539dfd9e .It is difficulty to distinguish > job when there are too much jobs . > A better way is to set the var of mapreduce.job.name .But set > mapreduce.job.name does not work! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12404) Orc ppd throws exception if types don't match
[
https://issues.apache.org/jira/browse/HIVE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-12404:
Fix Version/s: (was: 2.0.0)
> Orc ppd throws exception if types don't match
> -
>
> Key: HIVE-12404
> URL: https://issues.apache.org/jira/browse/HIVE-12404
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12404.patch
>
>
> When type of constant value and column doesn't match, currently Hive throws
> up.
> {code}
> java.lang.IllegalArgumentException: Wrong value class java.lang.Integer for
> BOOLEAN.LESS_THAN_EQUALS leaf
> at
> org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.(SearchArgumentImpl.java:63)
> at
> org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$BuilderImpl.lessThanEquals(SearchArgumentImpl.java:304)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.createLeaf(ConvertAstToSearchArg.java:277)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.createLeaf(ConvertAstToSearchArg.java:326)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.parse(ConvertAstToSearchArg.java:386)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.addChildren(ConvertAstToSearchArg.java:331)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.parse(ConvertAstToSearchArg.java:370)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.addChildren(ConvertAstToSearchArg.java:331)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.parse(ConvertAstToSearchArg.java:366)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.(ConvertAstToSearchArg.java:68)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.create(ConvertAstToSearchArg.java:417)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.createFromConf(ConvertAstToSearchArg.java:436)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$Context.(OrcInputFormat.java:484)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1121)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1207)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:369)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:481)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:160)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HIVE-12404) Orc ppd throws exception if types don't match
[
https://issues.apache.org/jira/browse/HIVE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan reopened HIVE-12404:
-
This is still an issue.
> Orc ppd throws exception if types don't match
> -
>
> Key: HIVE-12404
> URL: https://issues.apache.org/jira/browse/HIVE-12404
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12404.patch
>
>
> When type of constant value and column doesn't match, currently Hive throws
> up.
> {code}
> java.lang.IllegalArgumentException: Wrong value class java.lang.Integer for
> BOOLEAN.LESS_THAN_EQUALS leaf
> at
> org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.(SearchArgumentImpl.java:63)
> at
> org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$BuilderImpl.lessThanEquals(SearchArgumentImpl.java:304)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.createLeaf(ConvertAstToSearchArg.java:277)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.createLeaf(ConvertAstToSearchArg.java:326)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.parse(ConvertAstToSearchArg.java:386)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.addChildren(ConvertAstToSearchArg.java:331)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.parse(ConvertAstToSearchArg.java:370)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.addChildren(ConvertAstToSearchArg.java:331)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.parse(ConvertAstToSearchArg.java:366)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.(ConvertAstToSearchArg.java:68)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.create(ConvertAstToSearchArg.java:417)
> at
> org.apache.hadoop.hive.ql.io.sarg.ConvertAstToSearchArg.createFromConf(ConvertAstToSearchArg.java:436)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$Context.(OrcInputFormat.java:484)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1121)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1207)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:369)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:481)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:160)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12303) HCatRecordSerDe throw a IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019244#comment-15019244
]
Xiaowei Wang commented on HIVE-12303:
-
Could you give me some advice ? Thanks
> HCatRecordSerDe throw a IndexOutOfBoundsException
>
>
> Key: HIVE-12303
> URL: https://issues.apache.org/jira/browse/HIVE-12303
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.14.0, 1.2.1
>Reporter: Xiaowei Wang
>Assignee: Sushanth Sowmyan
> Fix For: 1.2.1
>
> Attachments: HIVE-12303.0.patch
>
>
> When access hive table using hcatlog in Pig,sometime it throws a exception !
> Exception
> {noformat}
> 2015-10-30 06:44:35,219 WARN [Thread-4] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
> at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:59)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:204)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IndexOutOfBoundsException: Index: 24, Size: 24
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeStruct(HCatRecordSerDe.java:175)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeList(HCatRecordSerDe.java:244)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:196)
> at
> org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> at
> org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> at
> org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:63)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12303) HCatRecordSerDe throw a IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019245#comment-15019245
]
Xiaowei Wang commented on HIVE-12303:
-
Could you give me some advice ? Thanks
> HCatRecordSerDe throw a IndexOutOfBoundsException
>
>
> Key: HIVE-12303
> URL: https://issues.apache.org/jira/browse/HIVE-12303
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.14.0, 1.2.1
>Reporter: Xiaowei Wang
>Assignee: Sushanth Sowmyan
> Fix For: 1.2.1
>
> Attachments: HIVE-12303.0.patch
>
>
> When access hive table using hcatlog in Pig,sometime it throws a exception !
> Exception
> {noformat}
> 2015-10-30 06:44:35,219 WARN [Thread-4] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
> at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:59)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:204)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IndexOutOfBoundsException: Index: 24, Size: 24
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeStruct(HCatRecordSerDe.java:175)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeList(HCatRecordSerDe.java:244)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:196)
> at
> org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> at
> org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> at
> org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:63)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12303) HCatRecordSerDe throw a IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019246#comment-15019246
]
Xiaowei Wang commented on HIVE-12303:
-
[~sushanth]
> HCatRecordSerDe throw a IndexOutOfBoundsException
>
>
> Key: HIVE-12303
> URL: https://issues.apache.org/jira/browse/HIVE-12303
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.14.0, 1.2.1
>Reporter: Xiaowei Wang
>Assignee: Sushanth Sowmyan
> Fix For: 1.2.1
>
> Attachments: HIVE-12303.0.patch
>
>
> When access hive table using hcatlog in Pig,sometime it throws a exception !
> Exception
> {noformat}
> 2015-10-30 06:44:35,219 WARN [Thread-4] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
> at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:59)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:204)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IndexOutOfBoundsException: Index: 24, Size: 24
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeStruct(HCatRecordSerDe.java:175)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeList(HCatRecordSerDe.java:244)
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:196)
> at
> org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> at
> org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> at
> org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:63)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12389) CompactionTxnHandler.cleanEmptyAbortedTxns() should safeguard against huge IN clauses
[ https://issues.apache.org/jira/browse/HIVE-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019248#comment-15019248 ] Jason Dere commented on HIVE-12389: --- Are the failures in TestSetUGIOnOnlyServer related? > CompactionTxnHandler.cleanEmptyAbortedTxns() should safeguard against huge IN > clauses > - > > Key: HIVE-12389 > URL: https://issues.apache.org/jira/browse/HIVE-12389 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12389.2.patch, HIVE-12389.patch > > > in extreme situations, due to misconfigurations, it may be possible to have > 100Ks or even 1Ms of aborted txns. > This causes delete from TXNS where txn_id in (...) to have a huge IN clause > and DB chokes. > Should use something like TxnHandler.TIMED_OUT_TXN_ABORT_BATCH_SIZE to break > up delete into multiple queries. (Incidentally the batch size should likely > be 1000, not 100, maybe even configurable). > On MySQL for example, it can cause query to fail with > bq. Packet for query is too large (9288598 > 1048576). You can change this > value on the server by setting the max_allowed_packet' variable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12338) Add webui to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15019250#comment-15019250 ] Szehon Ho commented on HIVE-12338: -- Jimmy explained this to me offline, also looks good from my end (seems a lot of code similar to other web-uis of hadoop ecosystem projects). I agree with Mohit that non-sql operations might be listed in a separate section. I think code-wise it makes sense for the sqlOperations to be calculated by hiveserver2.jsp webapp code rather than maintained by OperationManager (seems cleaner), I think it can be done in HIVE-12484. Other than that +1 > Add webui to HiveServer2 > > > Key: HIVE-12338 > URL: https://issues.apache.org/jira/browse/HIVE-12338 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-12338.1.patch, hs2-conf.png, hs2-logs.png, > hs2-metrics.png, hs2-webui.png > > > A web ui for HiveServer2 can show some useful information such as: > > 1. Sessions, > 2. Queries that are executing on the HS2, their states, starting time, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12020) Revert log4j2 xml configuration to properties based configuration
[ https://issues.apache.org/jira/browse/HIVE-12020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-12020: - Attachment: HIVE-12020.1.patch > Revert log4j2 xml configuration to properties based configuration > - > > Key: HIVE-12020 > URL: https://issues.apache.org/jira/browse/HIVE-12020 > Project: Hive > Issue Type: Sub-task > Components: Logging >Affects Versions: 2.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-12020.1.patch > > > Log4j 2.4 release brought back properties based configuration. We should > revert XML based configuration and use properties based configuration instead > (less verbose and will be similar to old log4j properties). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
