[jira] [Commented] (HIVE-12892) Add global change versioning to permanent functions in metastore
[ https://issues.apache.org/jira/browse/HIVE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139422#comment-15139422 ] Sergey Shelukhin commented on HIVE-12892: - Someone can drop a function and that changes the state of all functions. > Add global change versioning to permanent functions in metastore > > > Key: HIVE-12892 > URL: https://issues.apache.org/jira/browse/HIVE-12892 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12892.01.patch, HIVE-12892.02.patch, > HIVE-12892.03.patch, HIVE-12892.04.patch, HIVE-12892.05.nogen.patch, > HIVE-12892.05.patch, HIVE-12892.05.patch, HIVE-12892.nogen.patch, > HIVE-12892.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1608) use sequencefile as the default for storing intermediate results
[
https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139432#comment-15139432
]
Chaoyu Tang commented on HIVE-1608:
---
INSERT OVERWRITE [Local] DIRECTORY is actually not affected by this change, and
I have had some tests and verified it. It is because Hive already uses the
default tabledesc whose fileformat is hardcoded as "TextFile" for these cases.
See related code:
{code}
SemanticAnalyzer.java -- line 6523:
if (qb.getIsQuery()) {
String fileFormat = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVEQUERYRESULTFILEFORMAT);
table_desc = PlanUtils.getDefaultQueryOutputTableDesc(cols, colTypes,
fileFormat);
} else {
table_desc = PlanUtils.getDefaultTableDesc(qb.getDirectoryDesc(),
cols, colTypes);
}
---
PlanUtils.java -- 211, 224
public static TableDesc getDefaultTableDesc(String separatorCode,
String columns, String columnTypes, boolean lastColumnTakesRestOfTheLine)
{
return getTableDesc(LazySimpleSerDe.class, separatorCode, columns,
columnTypes, lastColumnTakesRestOfTheLine);
}
public static TableDesc getTableDesc(
Class serdeClass, String separatorCode,
String columns, String columnTypes, boolean lastColumnTakesRestOfTheLine,
boolean useDelimitedJSON) {
return getTableDesc(serdeClass, separatorCode, columns, columnTypes,
lastColumnTakesRestOfTheLine, useDelimitedJSON, "TextFile");
}
{code}
> use sequencefile as the default for storing intermediate results
>
>
> Key: HIVE-1608
> URL: https://issues.apache.org/jira/browse/HIVE-1608
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 0.7.0
>Reporter: Namit Jain
>Assignee: Brock Noland
> Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch,
> HIVE-1608.4.patch, HIVE-1608.patch
>
>
> The only argument for having a text file for storing intermediate results
> seems to be better debuggability.
> But, tailing a sequence file is possible, and it should be more space
> efficient
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-1608) use sequencefile as the default for storing intermediate results
[ https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-1608: -- Attachment: HIVE-1608.5.patch Update the patch for another just committed test vectorized_timestamp.q > use sequencefile as the default for storing intermediate results > > > Key: HIVE-1608 > URL: https://issues.apache.org/jira/browse/HIVE-1608 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Namit Jain >Assignee: Brock Noland > Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch, > HIVE-1608.4.patch, HIVE-1608.5.patch, HIVE-1608.patch > > > The only argument for having a text file for storing intermediate results > seems to be better debuggability. > But, tailing a sequence file is possible, and it should be more space > efficient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12941) Unexpected result when using MIN() on struct with NULL in first field
[
https://issues.apache.org/jira/browse/HIVE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-12941:
Attachment: HIVE-12941.3.patch
> Unexpected result when using MIN() on struct with NULL in first field
> -
>
> Key: HIVE-12941
> URL: https://issues.apache.org/jira/browse/HIVE-12941
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Jan-Erik Hedbom
>Assignee: Yongzhi Chen
> Attachments: HIVE-12941.1.patch, HIVE-12941.2.patch,
> HIVE-12941.3.patch
>
>
> Using MIN() on struct with NULL in first field of a row yields NULL as result.
> Example:
> select min(a) FROM (select 1 as a union all select 2 as a union all select
> cast(null as int) as a) tmp;
> OK
> _c0
> 1
> As expected. But if we wrap it in a struct:
> select min(a) FROM (select named_struct("field",1) as a union all select
> named_struct("field",2) as a union all select named_struct("field",cast(null
> as int)) as a) tmp;
> OK
> _c0
> NULL
> Using MAX() works as expected for structs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9862) Vectorized execution corrupts timestamp values
[
https://issues.apache.org/jira/browse/HIVE-9862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139454#comment-15139454
]
Matt McCline commented on HIVE-9862:
Committed to master.
> Vectorized execution corrupts timestamp values
> --
>
> Key: HIVE-9862
> URL: https://issues.apache.org/jira/browse/HIVE-9862
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 1.0.0
>Reporter: Nathan Howell
>Assignee: Matt McCline
> Fix For: 2.1.0
>
> Attachments: HIVE-9862.01.patch, HIVE-9862.02.patch,
> HIVE-9862.03.patch, HIVE-9862.04.patch, HIVE-9862.05.patch,
> HIVE-9862.06.patch, HIVE-9862.07.patch, HIVE-9862.08.patch, HIVE-9862.09.patch
>
>
> Timestamps in the future (year 2250?) and before ~1700 are silently corrupted
> in vectorized execution mode. Simple repro:
> {code}
> hive> DROP TABLE IF EXISTS test;
> hive> CREATE TABLE test(ts TIMESTAMP) STORED AS ORC;
> hive> INSERT INTO TABLE test VALUES ('-12-31 23:59:59');
> hive> SET hive.vectorized.execution.enabled = false;
> hive> SELECT MAX(ts) FROM test;
> -12-31 23:59:59
> hive> SET hive.vectorized.execution.enabled = true;
> hive> SELECT MAX(ts) FROM test;
> 1816-03-30 05:56:07.066277376
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13015) Update SLF4j version to 1.7.10
[
https://issues.apache.org/jira/browse/HIVE-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13015:
-
Assignee: Gopal V (was: Prasanth Jayachandran)
> Update SLF4j version to 1.7.10
> --
>
> Key: HIVE-13015
> URL: https://issues.apache.org/jira/browse/HIVE-13015
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Gopal V
> Attachments: HIVE-13015.1.patch
>
>
> In some of the recent test runs, we are seeing multiple bindings for SLF4j
> that causes issues with LOG4j2 logger.
> {code}
> SLF4J: Found binding in
> [jar:file:/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1454694331819_0001/container_e06_1454694331819_0001_01_02/app/install/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> {code}
> We have added explicit exclusions for slf4j-log4j12 but some library is
> pulling it transitively and it's getting packaged with hive libs. Also hive
> currently uses version 1.7.5 for slf4j. We should add dependency convergence
> for sl4fj and also remove packaging of slf4j-log4j12.*.jar
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13027) Async loggers for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139629#comment-15139629 ] Prasanth Jayachandran commented on HIVE-13027: -- [~gopalv] For using Async loggers, we need disruptor.jar in classpath. Currently disruptor.jar is pulled transitively by hbase-server. Do we need to package disruptor in hive-exec? > Async loggers for LLAP > -- > > Key: HIVE-13027 > URL: https://issues.apache.org/jira/browse/HIVE-13027 > Project: Hive > Issue Type: Improvement > Components: Logging >Affects Versions: 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-13027.1.patch > > > LOG4j2's async logger claims to have 6-68 times better performance than > synchronous logger. https://logging.apache.org/log4j/2.x/manual/async.html > We should use that for LLAP. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1608) use sequencefile as the default for storing intermediate results
[ https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139716#comment-15139716 ] Brock Noland commented on HIVE-1608: Thank you [~ctang.ma]! I came here to review so thanks to [~ashutoshc] as well. > use sequencefile as the default for storing intermediate results > > > Key: HIVE-1608 > URL: https://issues.apache.org/jira/browse/HIVE-1608 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Namit Jain >Assignee: Chaoyu Tang > Fix For: 2.1.0 > > Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch, > HIVE-1608.4.patch, HIVE-1608.5.patch, HIVE-1608.patch > > > The only argument for having a text file for storing intermediate results > seems to be better debuggability. > But, tailing a sequence file is possible, and it should be more space > efficient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1608) use sequencefile as the default for storing intermediate results
[ https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139718#comment-15139718 ] Chaoyu Tang commented on HIVE-1608: --- Updated the wiki and documented SequenceFile as the new default value for hive.query.result.fileformat since Hive 2.1.0. > use sequencefile as the default for storing intermediate results > > > Key: HIVE-1608 > URL: https://issues.apache.org/jira/browse/HIVE-1608 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Namit Jain >Assignee: Chaoyu Tang > Fix For: 2.1.0 > > Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch, > HIVE-1608.4.patch, HIVE-1608.5.patch, HIVE-1608.patch > > > The only argument for having a text file for storing intermediate results > seems to be better debuggability. > But, tailing a sequence file is possible, and it should be more space > efficient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate
[
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139388#comment-15139388
]
Hive QA commented on HIVE-9534:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786846/HIVE-9534.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10039 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_invalid_avg_syntax
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6920/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6920/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6920/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786846 - PreCommit-HIVE-TRUNK-Build
> incorrect result set for query that projects a windowed aggregate
> -
>
> Key: HIVE-9534
> URL: https://issues.apache.org/jira/browse/HIVE-9534
> Project: Hive
> Issue Type: Bug
> Components: PTF-Windowing
>Reporter: N Campbell
>Assignee: Aihua Xu
> Attachments: HIVE-9534.1.patch, HIVE-9534.2.patch, HIVE-9534.3.patch
>
>
> Result set returned by Hive has one row instead of 5
> {code}
> select avg(distinct tsint.csint) over () from tsint
> create table if not exists TSINT (RNUM int , CSINT smallint)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
> STORED AS TEXTFILE;
> 0|\N
> 1|-1
> 2|0
> 3|1
> 4|10
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12892) Add global change versioning to permanent functions in metastore
[ https://issues.apache.org/jira/browse/HIVE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139437#comment-15139437 ] Alan Gates commented on HIVE-12892: --- I'm +1 with committing as it and I'll just clean it up in HIVE-12927 since I need to change that anyway. But I don't understand your comment that "Someone can drop a function and that changes the state of all functions." How does dropping function A change the state of function B? I thought you were effectively producing version numbers for functions. Is that not what you're doing? > Add global change versioning to permanent functions in metastore > > > Key: HIVE-12892 > URL: https://issues.apache.org/jira/browse/HIVE-12892 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12892.01.patch, HIVE-12892.02.patch, > HIVE-12892.03.patch, HIVE-12892.04.patch, HIVE-12892.05.nogen.patch, > HIVE-12892.05.patch, HIVE-12892.05.patch, HIVE-12892.nogen.patch, > HIVE-12892.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12965) Insert overwrite local directory should perserve the overwritten directory permission
[ https://issues.apache.org/jira/browse/HIVE-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-12965: --- Attachment: HIVE-12965.1.patch Updated the patch based on the review and also uploaded to RB for review. [~xuefuz], please take a look and let me know if there is any question. Thanks > Insert overwrite local directory should perserve the overwritten directory > permission > - > > Key: HIVE-12965 > URL: https://issues.apache.org/jira/browse/HIVE-12965 > Project: Hive > Issue Type: Bug >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-12965.1.patch, HIVE-12965.patch > > > In Hive, "insert overwrite local directory" first deletes the overwritten > directory if exists, recreate a new one, then copy the files from src > directory to the new local directory. This process sometimes changes the > permissions of the to-be-overwritten local directory, therefore causing some > applications no more to be able to access its content. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12441) Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed
[
https://issues.apache.org/jira/browse/HIVE-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139710#comment-15139710
]
Hive QA commented on HIVE-12441:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786855/HIVE-12441.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9994 tests executed
*Failed tests:*
{noformat}
TestPerfCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6921/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6921/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6921/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786855 - PreCommit-HIVE-TRUNK-Build
> Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed
> --
>
> Key: HIVE-12441
> URL: https://issues.apache.org/jira/browse/HIVE-12441
> Project: Hive
> Issue Type: Bug
> Components: CLI, Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Wei Zheng
> Attachments: HIVE-12441.1.patch, HIVE-12441.2.patch
>
>
> recordValidTxns() is only needed if ACID tables are part of the query.
> Otherwise it's just overhead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12941) Unexpected result when using MIN() on struct with NULL in first field
[
https://issues.apache.org/jira/browse/HIVE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-12941:
Attachment: (was: HIVE-12941.3.patch)
> Unexpected result when using MIN() on struct with NULL in first field
> -
>
> Key: HIVE-12941
> URL: https://issues.apache.org/jira/browse/HIVE-12941
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Jan-Erik Hedbom
>Assignee: Yongzhi Chen
> Attachments: HIVE-12941.1.patch, HIVE-12941.2.patch
>
>
> Using MIN() on struct with NULL in first field of a row yields NULL as result.
> Example:
> select min(a) FROM (select 1 as a union all select 2 as a union all select
> cast(null as int) as a) tmp;
> OK
> _c0
> 1
> As expected. But if we wrap it in a struct:
> select min(a) FROM (select named_struct("field",1) as a union all select
> named_struct("field",2) as a union all select named_struct("field",cast(null
> as int)) as a) tmp;
> OK
> _c0
> NULL
> Using MAX() works as expected for structs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12941) Unexpected result when using MIN() on struct with NULL in first field
[
https://issues.apache.org/jira/browse/HIVE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-12941:
Attachment: HIVE-12941.3.patch
Re-attach the patch3 with comment fixes.
> Unexpected result when using MIN() on struct with NULL in first field
> -
>
> Key: HIVE-12941
> URL: https://issues.apache.org/jira/browse/HIVE-12941
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Jan-Erik Hedbom
>Assignee: Yongzhi Chen
> Attachments: HIVE-12941.1.patch, HIVE-12941.2.patch,
> HIVE-12941.3.patch
>
>
> Using MIN() on struct with NULL in first field of a row yields NULL as result.
> Example:
> select min(a) FROM (select 1 as a union all select 2 as a union all select
> cast(null as int) as a) tmp;
> OK
> _c0
> 1
> As expected. But if we wrap it in a struct:
> select min(a) FROM (select named_struct("field",1) as a union all select
> named_struct("field",2) as a union all select named_struct("field",cast(null
> as int)) as a) tmp;
> OK
> _c0
> NULL
> Using MAX() works as expected for structs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12892) Add global change versioning to permanent functions in metastore
[ https://issues.apache.org/jira/browse/HIVE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139424#comment-15139424 ] Sergey Shelukhin commented on HIVE-12892: - Yeah I didn't want to use sequences because of that. Are you ok with DN side of changes? It wouldn't make sense to commit all this table creating cruft and then drop it. For hbase metastore only, should be ok to fix in a subsequent JIRA. Is that a +1? :) > Add global change versioning to permanent functions in metastore > > > Key: HIVE-12892 > URL: https://issues.apache.org/jira/browse/HIVE-12892 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12892.01.patch, HIVE-12892.02.patch, > HIVE-12892.03.patch, HIVE-12892.04.patch, HIVE-12892.05.nogen.patch, > HIVE-12892.05.patch, HIVE-12892.05.patch, HIVE-12892.nogen.patch, > HIVE-12892.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Attachment: HIVE-12994.03.patch Reuploading, I had to make some changes after we upgraded to Calcite 1.6. > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Metastore, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1608) use sequencefile as the default for storing intermediate results
[ https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139442#comment-15139442 ] Ashutosh Chauhan commented on HIVE-1608: cool. +1 > use sequencefile as the default for storing intermediate results > > > Key: HIVE-1608 > URL: https://issues.apache.org/jira/browse/HIVE-1608 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Namit Jain >Assignee: Brock Noland > Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch, > HIVE-1608.4.patch, HIVE-1608.patch > > > The only argument for having a text file for storing intermediate results > seems to be better debuggability. > But, tailing a sequence file is possible, and it should be more space > efficient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9534) incorrect result set for query that projects a windowed aggregate
[
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-9534:
---
Attachment: HIVE-9534.4.patch
> incorrect result set for query that projects a windowed aggregate
> -
>
> Key: HIVE-9534
> URL: https://issues.apache.org/jira/browse/HIVE-9534
> Project: Hive
> Issue Type: Bug
> Components: PTF-Windowing
>Reporter: N Campbell
>Assignee: Aihua Xu
> Attachments: HIVE-9534.1.patch, HIVE-9534.2.patch, HIVE-9534.3.patch,
> HIVE-9534.4.patch
>
>
> Result set returned by Hive has one row instead of 5
> {code}
> select avg(distinct tsint.csint) over () from tsint
> create table if not exists TSINT (RNUM int , CSINT smallint)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
> STORED AS TEXTFILE;
> 0|\N
> 1|-1
> 2|0
> 3|1
> 4|10
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13029) NVDIMM support for LLAP Cache
[ https://issues.apache.org/jira/browse/HIVE-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-13029: --- Description: LLAP cache has been designed so that the cache can be offloaded easily to a pmem API without restart coherence. The tricky part about NVDIMMs are restart coherence, while most of the cache gains can be obtained without keeping state across refreshes, since LLAP is not the system of record, HDFS is. was: LLAP cache has been designed so that the cache can be offloaded easily to a pmem API without restart coherence. The tricky part about NVDIMMs are restart coherence, while most of the cache gains can be obtained without keeping state across refreshes. > NVDIMM support for LLAP Cache > - > > Key: HIVE-13029 > URL: https://issues.apache.org/jira/browse/HIVE-13029 > Project: Hive > Issue Type: New Feature > Components: llap >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Gopal V > > LLAP cache has been designed so that the cache can be offloaded easily to a > pmem API without restart coherence. > The tricky part about NVDIMMs are restart coherence, while most of the cache > gains can be obtained without keeping state across refreshes, since LLAP is > not the system of record, HDFS is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13030) Javadocs issue: Hive HCatalog build failed with IBM JDK 1.8 during Maven release
[
https://issues.apache.org/jira/browse/HIVE-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138773#comment-15138773
]
Devendra Vishwakarma commented on HIVE-13030:
-
Eventhough we are skipping the javadocs generation in maven.release command
like -
-X -Dresume=false release:prepare release:perform
-DworkingDirectory="${RELEASE_DIRECTORY}"
-Darguments="-Dmaven.javadoc.skip=true -DskipTests -Dgpg.skip=true
-Dmaven.javadoc.failOnError=false" -Phadoop-2,dist
But that is also not working to ignore javadoc generation during maven release.
> Javadocs issue: Hive HCatalog build failed with IBM JDK 1.8 during Maven
> release
>
>
> Key: HIVE-13030
> URL: https://issues.apache.org/jira/browse/HIVE-13030
> Project: Hive
> Issue Type: Bug
> Components: HCatalog, Hive, WebHCat
>Affects Versions: 1.2.1
> Environment: Hive 1.2.1 + IBM JDK 1.8 + s390x architecture
>Reporter: Devendra Vishwakarma
>Assignee: Devendra Vishwakarma
> Labels: HCatlog, Hive, IBM, Java, WebHCat, build, javadocs,
> maven, release
> Fix For: 1.2.1
>
>
> When building Hive with IBM JDK1.8, the maven release build is failing
> because of missing javadocs in Hive HCatolog webhcat module.
> All the errors are related to missing javadocs-
> 10:55:17 [INFO] [INFO] Hive HCatalog Webhcat .
> FAILURE [12.229s]
> 10:55:17 [INFO] [INFO] Hive HCatalog Streaming ...
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive HWI ..
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive ODBC .
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive Shims Aggregator .
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive TestUtils
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive Packaging
> SKIPPED
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [INFO] BUILD FAILURE
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [INFO] Total time: 4:10.477s
> 10:55:17 [INFO] [INFO] Finished at: Wed Feb 03 10:55:18 PST 2016
> 10:55:17 [INFO] [INFO] Final Memory: 79M/377M
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:2.4:jar (attach-javadocs) on
> project hive-webhcat: Error while creating archive:Exit code: 1 -
> /a/workspace//hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/AppConfig.java:353:
> warning: no @return
> 10:55:17 [INFO] [ERROR] public Collection hiveProps() {
> 10:55:17 [INFO] [ERROR] ^
> .
> .
> .
> .
> .
> There are lots of such errors coming in HCatlog package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12730) MetadataUpdater: provide a mechanism to edit the basic statistics of a table (or a partition)
[ https://issues.apache.org/jira/browse/HIVE-12730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12730: --- Attachment: HIVE-12730.07.patch > MetadataUpdater: provide a mechanism to edit the basic statistics of a table > (or a partition) > - > > Key: HIVE-12730 > URL: https://issues.apache.org/jira/browse/HIVE-12730 > Project: Hive > Issue Type: New Feature >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12730.01.patch, HIVE-12730.02.patch, > HIVE-12730.03.patch, HIVE-12730.04.patch, HIVE-12730.05.patch, > HIVE-12730.06.patch, HIVE-12730.07.patch > > > We would like to provide a way for developers/users to modify the numRows and > dataSize for a table/partition. Right now although they are part of the table > properties, they will be set to -1 when the task is not coming from a > statsTask. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12244) Refactoring code for avoiding of comparison of Strings and do comparison on Path
[
https://issues.apache.org/jira/browse/HIVE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138683#comment-15138683
]
Hive QA commented on HIVE-12244:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786806/HIVE-12244.6.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6917/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6917/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6917/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-6917/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 7cfeaef HIVE-12839: Upgrade Hive to Calcite 1.6 (Pengcheng
Xiong, reviewed by Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
+ git reset --hard origin/master
HEAD is now at 7cfeaef HIVE-12839: Upgrade Hive to Calcite 1.6 (Pengcheng
Xiong, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786806 - PreCommit-HIVE-TRUNK-Build
> Refactoring code for avoiding of comparison of Strings and do comparison on
> Path
>
>
> Key: HIVE-12244
> URL: https://issues.apache.org/jira/browse/HIVE-12244
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 0.13.0, 0.14.0, 1.0.0, 1.2.1
>Reporter: Alina Abramova
>Assignee: Alina Abramova
>Priority: Minor
> Labels: patch
> Fix For: 1.2.1
>
> Attachments: HIVE-12244.1.patch, HIVE-12244.2.patch,
> HIVE-12244.3.patch, HIVE-12244.4.patch, HIVE-12244.5.patch, HIVE-12244.6.patch
>
>
> In Hive often String is used for representation path and it causes new issues.
> We need to compare it with equals() but comparing Strings often is not right
> in terms comparing paths .
> I think if we use Path from org.apache.hadoop.fs we will avoid new problems
> in future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13030) Javadocs issue: Hive HCatalog build failed with IBM JDK 1.8 during Maven release
[
https://issues.apache.org/jira/browse/HIVE-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devendra Vishwakarma updated HIVE-13030:
Attachment: hive_build_javadocs_errors.txt
Added the detailed log (hive_build_javadocs_errors.txt).
> Javadocs issue: Hive HCatalog build failed with IBM JDK 1.8 during Maven
> release
>
>
> Key: HIVE-13030
> URL: https://issues.apache.org/jira/browse/HIVE-13030
> Project: Hive
> Issue Type: Bug
> Components: HCatalog, Hive, WebHCat
>Affects Versions: 1.2.1
> Environment: Hive 1.2.1 + IBM JDK 1.8 + s390x architecture
>Reporter: Devendra Vishwakarma
>Assignee: Devendra Vishwakarma
> Labels: HCatlog, Hive, IBM, Java, WebHCat, build, javadocs,
> maven, release
> Fix For: 1.2.1
>
> Attachments: hive_build_javadocs_errors.txt
>
>
> When building Hive with IBM JDK1.8, the maven release build is failing
> because of missing javadocs in Hive HCatolog webhcat module.
> All the errors are related to missing javadocs-
> 10:55:17 [INFO] [INFO] Hive HCatalog Webhcat .
> FAILURE [12.229s]
> 10:55:17 [INFO] [INFO] Hive HCatalog Streaming ...
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive HWI ..
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive ODBC .
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive Shims Aggregator .
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive TestUtils
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive Packaging
> SKIPPED
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [INFO] BUILD FAILURE
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [INFO] Total time: 4:10.477s
> 10:55:17 [INFO] [INFO] Finished at: Wed Feb 03 10:55:18 PST 2016
> 10:55:17 [INFO] [INFO] Final Memory: 79M/377M
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:2.4:jar (attach-javadocs) on
> project hive-webhcat: Error while creating archive:Exit code: 1 -
> /a/workspace//hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/AppConfig.java:353:
> warning: no @return
> 10:55:17 [INFO] [ERROR] public Collection hiveProps() {
> 10:55:17 [INFO] [ERROR] ^
> .
> .
> .
> .
> .
> There are lots of such errors coming in HCatlog package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11749) Deadlock of fetching InputFormat table when multiple root stage
[ https://issues.apache.org/jira/browse/HIVE-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated HIVE-11749: -- Attachment: HIVE-11749.01.patch > Deadlock of fetching InputFormat table when multiple root stage > --- > > Key: HIVE-11749 > URL: https://issues.apache.org/jira/browse/HIVE-11749 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Ryu Kobayashi >Assignee: Kai Sasaki > Attachments: HIVE-11749.00.patch, HIVE-11749.01.patch, > HIVE-11749.stack-tarace.txt > > > But not always, to deadlock when it run the query. Environment are as follows: > * Hadoop 2.6.0 > * Hive 0.13 > * JDK 1.7.0_79 > It will attach the stack trace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11749) Deadlock of fetching InputFormat table when multiple root stage
[ https://issues.apache.org/jira/browse/HIVE-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138800#comment-15138800 ] Kai Sasaki commented on HIVE-11749: --- [~tasanuma0829] Thank you for pinging. We fixed the dead lock problem with the latest patch that was updated. ([HIVE-11749.02.patch|https://issues.apache.org/jira/secure/attachment/12787053/HIVE-11749.01.patch]). [~sershe] [~gopalv] Could you review this? > Deadlock of fetching InputFormat table when multiple root stage > --- > > Key: HIVE-11749 > URL: https://issues.apache.org/jira/browse/HIVE-11749 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Ryu Kobayashi >Assignee: Kai Sasaki > Attachments: HIVE-11749.00.patch, HIVE-11749.01.patch, > HIVE-11749.stack-tarace.txt > > > But not always, to deadlock when it run the query. Environment are as follows: > * Hadoop 2.6.0 > * Hive 0.13 > * JDK 1.7.0_79 > It will attach the stack trace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1608) use sequencefile as the default for storing intermediate results
[
https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138909#comment-15138909
]
Hive QA commented on HIVE-1608:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786826/HIVE-1608.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10024 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-unionDistinct_1.q-insert_values_non_partitioned.q-selectDistinctStar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_folder_predicate
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_llap_nullscan
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6918/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6918/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6918/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786826 - PreCommit-HIVE-TRUNK-Build
> use sequencefile as the default for storing intermediate results
>
>
> Key: HIVE-1608
> URL: https://issues.apache.org/jira/browse/HIVE-1608
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 0.7.0
>Reporter: Namit Jain
>Assignee: Brock Noland
> Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch,
> HIVE-1608.patch
>
>
> The only argument for having a text file for storing intermediate results
> seems to be better debuggability.
> But, tailing a sequence file is possible, and it should be more space
> efficient
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13030) Javadocs issue: Hive HCatalog build failed with IBM JDK 1.8 during Maven release
[
https://issues.apache.org/jira/browse/HIVE-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devendra Vishwakarma updated HIVE-13030:
Attachment: HIVE-13030.patch
Attached patch [HIVE-13030.patch] having missing javadocs which causes maven
release build failed.
> Javadocs issue: Hive HCatalog build failed with IBM JDK 1.8 during Maven
> release
>
>
> Key: HIVE-13030
> URL: https://issues.apache.org/jira/browse/HIVE-13030
> Project: Hive
> Issue Type: Bug
> Components: HCatalog, Hive, WebHCat
>Affects Versions: 1.2.1
> Environment: Hive 1.2.1 + IBM JDK 1.8 + s390x architecture
>Reporter: Devendra Vishwakarma
>Assignee: Devendra Vishwakarma
> Labels: HCatlog, Hive, IBM, Java, WebHCat, build, javadocs,
> maven, release
> Fix For: 1.2.1
>
> Attachments: HIVE-13030.patch, hive_build_javadocs_errors.txt
>
>
> When building Hive with IBM JDK1.8, the maven release build is failing
> because of missing javadocs in Hive HCatolog webhcat module.
> All the errors are related to missing javadocs-
> 10:55:17 [INFO] [INFO] Hive HCatalog Webhcat .
> FAILURE [12.229s]
> 10:55:17 [INFO] [INFO] Hive HCatalog Streaming ...
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive HWI ..
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive ODBC .
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive Shims Aggregator .
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive TestUtils
> SKIPPED
> 10:55:17 [INFO] [INFO] Hive Packaging
> SKIPPED
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [INFO] BUILD FAILURE
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [INFO] Total time: 4:10.477s
> 10:55:17 [INFO] [INFO] Finished at: Wed Feb 03 10:55:18 PST 2016
> 10:55:17 [INFO] [INFO] Final Memory: 79M/377M
> 10:55:17 [INFO] [INFO]
>
> 10:55:17 [INFO] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:2.4:jar (attach-javadocs) on
> project hive-webhcat: Error while creating archive:Exit code: 1 -
> /a/workspace//hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/AppConfig.java:353:
> warning: no @return
> 10:55:17 [INFO] [ERROR] public Collection hiveProps() {
> 10:55:17 [INFO] [ERROR] ^
> .
> .
> .
> .
> .
> There are lots of such errors coming in HCatlog package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12965) Insert overwrite local directory should perserve the overwritten directory permission
[ https://issues.apache.org/jira/browse/HIVE-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139111#comment-15139111 ] Xuefu Zhang commented on HIVE-12965: Patch looks good. Some minor comments on RB. If you need to submit a patch, the if-else block (isDfsDir()) is relative big and it might be more readable if we can refactor them into helper methods. > Insert overwrite local directory should perserve the overwritten directory > permission > - > > Key: HIVE-12965 > URL: https://issues.apache.org/jira/browse/HIVE-12965 > Project: Hive > Issue Type: Bug >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-12965.patch > > > In Hive, "insert overwrite local directory" first deletes the overwritten > directory if exists, recreate a new one, then copy the files from src > directory to the new local directory. This process sometimes changes the > permissions of the to-be-overwritten local directory, therefore causing some > applications no more to be able to access its content. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12988) Improve dynamic partition loading IV
[ https://issues.apache.org/jira/browse/HIVE-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12988: Attachment: HIVE-12988.3.patch > Improve dynamic partition loading IV > > > Key: HIVE-12988 > URL: https://issues.apache.org/jira/browse/HIVE-12988 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 1.2.0, 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12988.2.patch, HIVE-12988.2.patch, > HIVE-12988.3.patch, HIVE-12988.patch > > > Parallelize copyFiles() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12987) Add metrics for HS2 active users and SQL operations
[ https://issues.apache.org/jira/browse/HIVE-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139142#comment-15139142 ] Aihua Xu commented on HIVE-12987: - +1 on the new patch. Of course we need to wait for the new build, but the tests above seem not related, right? > Add metrics for HS2 active users and SQL operations > --- > > Key: HIVE-12987 > URL: https://issues.apache.org/jira/browse/HIVE-12987 > Project: Hive > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-12987.1.patch, HIVE-12987.2.patch, > HIVE-12987.2.patch, HIVE-12987.3.patch, HIVE-12987.3.patch, HIVE-12987.4.patch > > > HIVE-12271 added metrics for all HS2 operations. Sometimes, users are also > interested in metrics just for SQL operations. > It is useful to track active user count as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12941) Unexpected result when using MIN() on struct with NULL in first field
[
https://issues.apache.org/jira/browse/HIVE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139137#comment-15139137
]
Aihua Xu commented on HIVE-12941:
-
The patch in general looks good. I would suggest to use a enum {Min, Max} to
treat NULL as Min or Max in comparison instead of a boolean.
> Unexpected result when using MIN() on struct with NULL in first field
> -
>
> Key: HIVE-12941
> URL: https://issues.apache.org/jira/browse/HIVE-12941
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Jan-Erik Hedbom
>Assignee: Yongzhi Chen
> Attachments: HIVE-12941.1.patch, HIVE-12941.2.patch
>
>
> Using MIN() on struct with NULL in first field of a row yields NULL as result.
> Example:
> select min(a) FROM (select 1 as a union all select 2 as a union all select
> cast(null as int) as a) tmp;
> OK
> _c0
> 1
> As expected. But if we wrap it in a struct:
> select min(a) FROM (select named_struct("field",1) as a union all select
> named_struct("field",2) as a union all select named_struct("field",cast(null
> as int)) as a) tmp;
> OK
> _c0
> NULL
> Using MAX() works as expected for structs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13021) GenericUDAFEvaluator.isEstimable(agg) always returns false
[ https://issues.apache.org/jira/browse/HIVE-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zadoroshnyak updated HIVE-13021: --- Priority: Critical (was: Major) > GenericUDAFEvaluator.isEstimable(agg) always returns false > -- > > Key: HIVE-13021 > URL: https://issues.apache.org/jira/browse/HIVE-13021 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 1.2.1 >Reporter: Sergey Zadoroshnyak >Assignee: Gopal V >Priority: Critical > Labels: Performance > > GenericUDAFEvaluator.isEstimable(agg) always returns false, because > annotation AggregationType has default RetentionPolicy.CLASS and cannot be > retained by the VM at run time. > As result estimate method will never be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-1608) use sequencefile as the default for storing intermediate results
[ https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-1608: -- Attachment: HIVE-1608.4.patch Fixed failures to new llap and tez tests llap_nullscan.q, also the folder_predicate.q. Other failures are not related to this patch. [~brocknoland], [~ashutoshc] could you review the patch? The changed tests will become stale very soon. Thanks. > use sequencefile as the default for storing intermediate results > > > Key: HIVE-1608 > URL: https://issues.apache.org/jira/browse/HIVE-1608 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Namit Jain >Assignee: Brock Noland > Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch, > HIVE-1608.4.patch, HIVE-1608.patch > > > The only argument for having a text file for storing intermediate results > seems to be better debuggability. > But, tailing a sequence file is possible, and it should be more space > efficient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[
https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139090#comment-15139090
]
Hive QA commented on HIVE-12994:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786966/HIVE-12994.02.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 34 failed/errored test(s), 10041 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-vector_decimal_round.q-cbo_windowing.q-tez_schema_evolution.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_colname
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_resolution
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lineage2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lineage3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_order_null
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcs
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ptfgroupbyjoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_quotedid_basic
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in_having
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_unqualcolumnrefs
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_6_subq
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_windowing_navfn
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_windowing_streaming
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_windowing_windowspec3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_subquery_in
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_windowing_gby
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query20
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query51
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query67
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query70
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query89
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query98
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_resolution
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_subquery_in
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union_remove_6_subq
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6919/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6919/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6919/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 34 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786966 - PreCommit-HIVE-TRUNK-Build
> Implement support for NULLS FIRST/NULLS LAST
>
>
> Key: HIVE-12994
> URL: https://issues.apache.org/jira/browse/HIVE-12994
> Project: Hive
> Issue Type: New Feature
> Components: CBO, Metastore, Parser, Serializers/Deserializers
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch,
> HIVE-12994.patch
>
>
> From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to
> determine whether nulls appear before or after non-null data values when the
> ORDER BY clause is used.
> SQL standard does not specify the behavior by default. Currently in Hive,
> null values sort as if lower than any non-null value; that is, NULLS FIRST is
> the default for ASC order, and NULLS LAST for DESC order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-1608) use sequencefile as the default for storing intermediate results
[ https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139113#comment-15139113 ] Ashutosh Chauhan commented on HIVE-1608: As noted earlier in thread this will be an incompatible change for INSERT OVERWRITE DIRECTORY case. Seems like your patch doesnt handle that. > use sequencefile as the default for storing intermediate results > > > Key: HIVE-1608 > URL: https://issues.apache.org/jira/browse/HIVE-1608 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Namit Jain >Assignee: Brock Noland > Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch, > HIVE-1608.4.patch, HIVE-1608.patch > > > The only argument for having a text file for storing intermediate results > seems to be better debuggability. > But, tailing a sequence file is possible, and it should be more space > efficient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10294) ParseException issue (Failed to recognize predicate 'user') [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139200#comment-15139200
]
Sivashankar commented on HIVE-10294:
In the Query user backtick character . `user` instead of user. Also in json
records for the key that starts with underscore( _ )
> ParseException issue (Failed to recognize predicate 'user') [Spark Branch]
> --
>
> Key: HIVE-10294
> URL: https://issues.apache.org/jira/browse/HIVE-10294
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Xin Hao
>
> ParseException (Failed to recognize predicate 'user') occurred when we run
> some Hive query (e.g. BigBench case Q03).
> By the way, if we change all the 'user' to 'user2', the parsing could be
> passed.
> (1) Exception Details:
> {noformat}
> FailedPredicateException(identifier,{useSQL11ReservedKeywordsForIdentifier()}?)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:10912)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:45479)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectItem(HiveParser_SelectClauseParser.java:2933)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectList(HiveParser_SelectClauseParser.java:1367)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectClause(HiveParser_SelectClauseParser.java:1122)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectClause(HiveParser.java:45475)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:41136)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:41043)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:40059)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39929)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:5295)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:3731)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1865)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1512)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45556)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.singleFromStatement(HiveParser.java:40666)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromStatement(HiveParser.java:40389)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:40044)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39929)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:5295)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:3731)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1865)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1512)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45556)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:41157)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40876)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:40059)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39929)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1574)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.explainStatement(HiveParser.java:1269)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1070)
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:202)
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:397)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1115)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1163)
> at
[jira] [Commented] (HIVE-6050) Newer versions of JDBC driver does not work with older HiveServer2
[
https://issues.apache.org/jira/browse/HIVE-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139249#comment-15139249
]
sanjiv singh commented on HIVE-6050:
Any update on this ..we are facing same issue with 1.2.1.
> Newer versions of JDBC driver does not work with older HiveServer2
> --
>
> Key: HIVE-6050
> URL: https://issues.apache.org/jira/browse/HIVE-6050
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, JDBC
>Affects Versions: 0.13.0
>Reporter: Szehon Ho
>Priority: Blocker
>
> HiveServer2 instance has to be upgraded before the JDBC drivers used by
> applications are upgraded. If jdbc drivers are updated before HiveServer2 is
> upgraded it will not be functional.
> Connect from JDBC driver of Hive 0.13 (TProtocolVersion=v4) to HiveServer2 of
> Hive 0.10 (TProtocolVersion=v1), will return the following exception:
> {noformat}
> java.sql.SQLException: Could not establish connection to
> jdbc:hive2://localhost:1/default: Required field 'client_protocol' is
> unset! Struct:TOpenSessionReq(client_protocol:null)
> at
> org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:336)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:158)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
> at java.sql.DriverManager.getConnection(DriverManager.java:571)
> at java.sql.DriverManager.getConnection(DriverManager.java:187)
> at
> org.apache.hive.jdbc.MyTestJdbcDriver2.getConnection(MyTestJdbcDriver2.java:73)
> at
> org.apache.hive.jdbc.MyTestJdbcDriver2.init(MyTestJdbcDriver2.java:49)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:187)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:236)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:233)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:523)
> at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1063)
> at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:914)
> Caused by: org.apache.thrift.TApplicationException: Required field
> 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null)
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
> at
> org.apache.hive.service.cli.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:160)
> at
> org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:147)
> at
> org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:327)
> ... 37 more
> {noformat}
> On code analysis, it looks like the 'client_protocol' scheme is a ThriftEnum,
> which doesn't seem to be backward-compatible. Look at the code path in the
> generated file 'TOpenSessionReq.java', method
> TOpenSessionReqStandardScheme.read():
> 1. The method will call 'TProtocolVersion.findValue()' on the thrift
> protocol's byte stream, which returns null if the client is sending an enum
> value unknown to the server. (v4 is unknown to server)
> 2. The method will then call struct.validate(), which will throw the above
> exception because of null version.
> So doesn't look like the current backward-compatibility scheme will work.
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HIVE-11749) Deadlock of fetching InputFormat table when multiple root stage
[ https://issues.apache.org/jira/browse/HIVE-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139790#comment-15139790 ] Sergey Shelukhin commented on HIVE-11749: - Hmm, I'm not very familiar with this code. This is MR-only, right? The patch makes sense (I wonder if all partition objects actually have different tabledesc objects - if not, reference equality would be enough and the problem would go away; otherwise, I wonder if comparing db + table names is enough). [~hagleitn] any input? git blame points at you ;) Otherwise +1 > Deadlock of fetching InputFormat table when multiple root stage > --- > > Key: HIVE-11749 > URL: https://issues.apache.org/jira/browse/HIVE-11749 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Ryu Kobayashi >Assignee: Kai Sasaki > Attachments: HIVE-11749.00.patch, HIVE-11749.01.patch, > HIVE-11749.stack-tarace.txt > > > But not always, to deadlock when it run the query. Environment are as follows: > * Hadoop 2.6.0 > * Hive 0.13 > * JDK 1.7.0_79 > It will attach the stack trace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-11749) Deadlock of fetching InputFormat table when multiple root stage
[ https://issues.apache.org/jira/browse/HIVE-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139790#comment-15139790 ] Sergey Shelukhin edited comment on HIVE-11749 at 2/9/16 9:23 PM: - Hmm, I'm not very familiar with this code. This multi-root situation is MR-only, right? The patch makes sense (I wonder if all partition objects actually have different tabledesc objects - if not, reference equality would be enough and the problem would go away; otherwise, I wonder if comparing db + table names is enough). [~hagleitn] any input? git blame points at you ;) Otherwise +1 was (Author: sershe): Hmm, I'm not very familiar with this code. This is MR-only, right? The patch makes sense (I wonder if all partition objects actually have different tabledesc objects - if not, reference equality would be enough and the problem would go away; otherwise, I wonder if comparing db + table names is enough). [~hagleitn] any input? git blame points at you ;) Otherwise +1 > Deadlock of fetching InputFormat table when multiple root stage > --- > > Key: HIVE-11749 > URL: https://issues.apache.org/jira/browse/HIVE-11749 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Ryu Kobayashi >Assignee: Kai Sasaki > Attachments: HIVE-11749.00.patch, HIVE-11749.01.patch, > HIVE-11749.stack-tarace.txt > > > But not always, to deadlock when it run the query. Environment are as follows: > * Hadoop 2.6.0 > * Hive 0.13 > * JDK 1.7.0_79 > It will attach the stack trace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12987) Add metrics for HS2 active users and SQL operations
[
https://issues.apache.org/jira/browse/HIVE-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139870#comment-15139870
]
Hive QA commented on HIVE-12987:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786887/HIVE-12987.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9768 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.createTable
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLockRetryLimit
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.updateSelectUpdate
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6922/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6922/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6922/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786887 - PreCommit-HIVE-TRUNK-Build
> Add metrics for HS2 active users and SQL operations
> ---
>
> Key: HIVE-12987
> URL: https://issues.apache.org/jira/browse/HIVE-12987
> Project: Hive
> Issue Type: Task
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: HIVE-12987.1.patch, HIVE-12987.2.patch,
> HIVE-12987.2.patch, HIVE-12987.3.patch, HIVE-12987.3.patch, HIVE-12987.4.patch
>
>
> HIVE-12271 added metrics for all HS2 operations. Sometimes, users are also
> interested in metrics just for SQL operations.
> It is useful to track active user count as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12254) Improve logging with yarn/hdfs
[
https://issues.apache.org/jira/browse/HIVE-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139912#comment-15139912
]
Sergey Shelukhin commented on HIVE-12254:
-
{noformat}
+String logPrefix = getHiveConf().getLogIdVar(sessionState.getSessionId());
+ShimLoader.getHadoopShims().setHadoopCallerContext(logPrefix);
{noformat}
Should this call the ...SessionContext method?
Otherwise looks good. +1 can be fixed on commit
> Improve logging with yarn/hdfs
> --
>
> Key: HIVE-12254
> URL: https://issues.apache.org/jira/browse/HIVE-12254
> Project: Hive
> Issue Type: Bug
> Components: Shims
>Affects Versions: 1.2.1
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-12254.1.patch, HIVE-12254.2.patch
>
>
> In extension to HIVE-12249, adding info for Yarn/HDFS as well. Both
> HIVE-12249 and HDFS-9184 are required (and upgraded in hive for the HDFS
> issue) before this can be resolved.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12965) Insert overwrite local directory should perserve the overwritten directory permission
[ https://issues.apache.org/jira/browse/HIVE-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139743#comment-15139743 ] Xuefu Zhang commented on HIVE-12965: +1 > Insert overwrite local directory should perserve the overwritten directory > permission > - > > Key: HIVE-12965 > URL: https://issues.apache.org/jira/browse/HIVE-12965 > Project: Hive > Issue Type: Bug >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-12965.1.patch, HIVE-12965.2.patch, HIVE-12965.patch > > > In Hive, "insert overwrite local directory" first deletes the overwritten > directory if exists, recreate a new one, then copy the files from src > directory to the new local directory. This process sometimes changes the > permissions of the to-be-overwritten local directory, therefore causing some > applications no more to be able to access its content. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12965) Insert overwrite local directory should perserve the overwritten directory permission
[ https://issues.apache.org/jira/browse/HIVE-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-12965: --- Attachment: HIVE-12965.2.patch uploaded a revised patch. [~xuefuz], please review it. Thanks > Insert overwrite local directory should perserve the overwritten directory > permission > - > > Key: HIVE-12965 > URL: https://issues.apache.org/jira/browse/HIVE-12965 > Project: Hive > Issue Type: Bug >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-12965.1.patch, HIVE-12965.2.patch, HIVE-12965.patch > > > In Hive, "insert overwrite local directory" first deletes the overwritten > directory if exists, recreate a new one, then copy the files from src > directory to the new local directory. This process sometimes changes the > permissions of the to-be-overwritten local directory, therefore causing some > applications no more to be able to access its content. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12878) Support Vectorization for TEXTFILE and other formats
[
https://issues.apache.org/jira/browse/HIVE-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139766#comment-15139766
]
Matt McCline commented on HIVE-12878:
-
I just went through the 1253 test failures to filter out the expected
"Execution mode: vectorized", statistics differences, etc.
Here are the query wrong results and test failures. A rather stunning amount.
{code}
TestCliDriver
o Wrong Results:
• add_part_multiple
• alter_partition_coltype
• alter_varchar2
• analyze_tbl_part
• auto_join18
• auto_join18_multi_distinct
• avro_schema_evolution_native
• avro_timestamp
• bucket_groupby
• cbo_const
• cbo_rp_lineage2
• cbo_rp_union
• cbo_rp_views
• cbo_rp_windowing
• cbo_union
• cbo_views
• cbo_windowing
• complex_alias
• constprog_type
• correlationoptimizer14
• correlationoptimizer2
• correlationoptimizer8
• ctas_colname
• custom_input_output_format
• date_1
• date_3
• date_udf
• decimal_1
• decimal_2
• empty_join
• filter_join_breaktask2
• groupby_duplicate_key
• groupby_grouping_window
• groupby_sort_10
• insert_into1
• interval_arithmetic
• join18
• join18_multi_distinct
• lineage2
• mapjoin_test_outer
• metadata_only_queries
• metadata_only_queries_with_filters
• non_ascii_literal
• orc_dictionary_threshold
• orc_diff_part_cols
• orc_empty_strings
• orc_file_dump
• orc_int_type_promotion
• orc_predicate_pushdown
• offset_limit_global_optimizer
• parquet_ppd_decimal
• parquet_predicate_pushdown
• partcols1
• partition_date
• partition_date2
• partition_multilevels
• partition_timestamp
• partition_timestamp2
• partition_varchar1
• partition_wise_fileformat2
• ppr_pushdown2
• rcfile_null_value
• selectDistinctStar
• special_characters_in_tabnames_1
• stats1
• str_to_map
• temp_table_windowing_expressions
• test_boolean_whereclause
• timestamp_3
• timestamp_lazy
• timestamp_udf
• truncate_column
• truncate_column_merge
• udf_context_aware
• udf_get_json_object
• udf_length
• udf_printf
• udf_round_2
• udtf_json_tuple
• union6
• union34
• unionDistinct_1
• vector_binary_join_groupby
• vector_data_types
• vector_decimal_1
• vector_decimal_2
• vector_orderby_5
• windowing_distinct
• windowing_expressions
• windowing_multipartitioning
• windowing_navfn
• windowing_rank
o Failures:
• auto_join_reordering_values
• auto_sortmerge_join_1
• auto_sortmerge_join_14
• auto_sortmerge_join_2
• auto_sortmerge_join_3
• auto_sortmerge_join_4
• auto_sortmerge_join_5
• auto_sortmerge_join_6
• auto_sortmerge_join_7
• auto_sortmerge_join_9
• bucketsortoptimize_insert_2
• bucketsortoptimize_insert_4
• bucketsortoptimize_insert_5
• join42
• join_filters
• mapjoin1
• orc_min_max
• partition_wise_fileformat16
• ppd_union_view
• skewjoin
• vector_elt
TestContribNegativeCliDriver
o Wrong Results:
o Failures:
• case_with_row_sequence
TestHBaseCliDriver
o Wrong Results:
• hbase_single_sourced_multi_insert
o Failures:
TestMiniLlapCliDriver
o Wrong Results:
• hybridgrace_hashjoin_1
• hybridgrace_hashjoin_2
• tez_join_tests
• tez_union_decimal
o Failures:
• bucket_map_join_tez1
• tez_bmj_schema_evolution
• tez_smb_main
• TestMiniSparkOnYarnCliDriver
o Wrong Results:
• schemaAuthority2
• vector_outer_join1
• vector_outer_join2
• vector_outer_join3
• vector_outer_join4
o Failures:
• bucketmapjoin7
TestMiniTezCliDriver
o Wrong Results:
• cbo_simple_select
• cbo_union
• cbo_views
• cbo_windowing
• custom_input_output_format
• empty_join
• filter_join_breaktask2
• hybridgrace_hashjoin_1
• hybridgrace_hashjoin_2
• insert_into1
• mergejoin
• metadata_queries_only
• metadata_queries_only_with_filters
• selectDistinctStar
• select_dummy_source
• tez_join_tests
• tez_union_decimal
• union6
• unionDistinct_1
• vector_binary_join_groupby
• vector_data_types
• vector_decimal_1
• vector_decimal_2
• vector_outer_join1
• vector_outer_join2
• vector_outer_join3
• vector_outer_join4
• vector_orderby_5
• vector_when_case_null
• vectorized_date_funcs
o Failures:
•
[jira] [Commented] (HIVE-10187) Avro backed tables don't handle cyclical or recursive records
[ https://issues.apache.org/jira/browse/HIVE-10187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139862#comment-15139862 ] Anthony Hsu commented on HIVE-10187: New patch LGTM. > Avro backed tables don't handle cyclical or recursive records > - > > Key: HIVE-10187 > URL: https://issues.apache.org/jira/browse/HIVE-10187 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Mark Wagner >Assignee: Mark Wagner > Attachments: HIVE-10187.1.patch, HIVE-10187.2.patch, > HIVE-10187.3.patch, HIVE-10187.4.patch, HIVE-10187.5.patch, > HIVE-10187.demo.patch > > > [HIVE-7653] changed the Avro SerDe to make it generate TypeInfos even for > recursive/cyclical schemas. However, any attempt to serialize data which > exploits that ability results in silently dropped fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.4.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch, > HIVE-12963.3.patch, HIVE-12963.4.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with querie: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12441) Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed
[ https://issues.apache.org/jira/browse/HIVE-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139854#comment-15139854 ] Wei Zheng commented on HIVE-12441: -- This essentially will avoid having too much unnecessary logging for non-ACID table, especially in extreme cases where there are hundreds of thousands of transactions > Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed > -- > > Key: HIVE-12441 > URL: https://issues.apache.org/jira/browse/HIVE-12441 > Project: Hive > Issue Type: Bug > Components: CLI, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12441.1.patch, HIVE-12441.2.patch > > > recordValidTxns() is only needed if ACID tables are part of the query. > Otherwise it's just overhead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13032) Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization
[ https://issues.apache.org/jira/browse/HIVE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-13032: - Attachment: HIVE-13032.1.patch > Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization > -- > > Key: HIVE-13032 > URL: https://issues.apache.org/jira/browse/HIVE-13032 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-13032.1.patch > > > HIVE-12497 removed HADOOP_CLIENT_OPTS as it slowed down cli launch time. But > it leads to log4j2 not being initialized when using services other than CLI. > Other services like metastore, schematool etc. rely on log4j to initialize > the logging based on the presence of log4j2.properties file in the classpath. > If we use the standard name for log4j configuration file (log4j2.properties) > then automatic initialization will happen. If not, we have to tell log4j to > look for specific properties file. This is done via -Dlog4j.configurationFile > system property. If we pass this system property via HADOOP_CLIENT_OPTS then > all hive services will have logging initialized properly. > In HIVE-12497, the problem was we had HADOOP_CLIENT_OPTS at the top of the > script. As a result, hadoop and hbase commands tries to initialize logging > which took long time slowing down the startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12857) LLAP: modify the decider to allow using LLAP with whitelisted UDFs
[
https://issues.apache.org/jira/browse/HIVE-12857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139874#comment-15139874
]
Hive QA commented on HIVE-12857:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786906/HIVE-12857.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6924/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6924/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6924/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-6924/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
dcf74c4..a285de0 master -> origin/master
+ git reset --hard HEAD
HEAD is now at dcf74c4 HIVE-13016: ORC FileDump recovery utility fails in
Windows (Prasanth Jayachandran reviewed by Jason Dere)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
+ git reset --hard origin/master
HEAD is now at a285de0 HIVE-12993: user and password supplied from URL is
overwritten by the empty user and password of the JDBC connection string when
it's calling from beeline (Aihua Xu, reviewed by Yongzhi Chen)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786906 - PreCommit-HIVE-TRUNK-Build
> LLAP: modify the decider to allow using LLAP with whitelisted UDFs
> --
>
> Key: HIVE-12857
> URL: https://issues.apache.org/jira/browse/HIVE-12857
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-12857.01.patch, HIVE-12857.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12441) Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed
[ https://issues.apache.org/jira/browse/HIVE-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139871#comment-15139871 ] Alan Gates commented on HIVE-12441: --- Ok, +1 > Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed > -- > > Key: HIVE-12441 > URL: https://issues.apache.org/jira/browse/HIVE-12441 > Project: Hive > Issue Type: Bug > Components: CLI, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12441.1.patch, HIVE-12441.2.patch > > > recordValidTxns() is only needed if ACID tables are part of the query. > Otherwise it's just overhead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12892) Add global change versioning to permanent functions in metastore
[ https://issues.apache.org/jira/browse/HIVE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-12892: Issue Type: Sub-task (was: Bug) Parent: HIVE-12852 > Add global change versioning to permanent functions in metastore > > > Key: HIVE-12892 > URL: https://issues.apache.org/jira/browse/HIVE-12892 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.1.0 > > Attachments: HIVE-12892.01.patch, HIVE-12892.02.patch, > HIVE-12892.03.patch, HIVE-12892.04.patch, HIVE-12892.05.nogen.patch, > HIVE-12892.05.patch, HIVE-12892.05.patch, HIVE-12892.nogen.patch, > HIVE-12892.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12441) Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed
[ https://issues.apache.org/jira/browse/HIVE-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139725#comment-15139725 ] Wei Zheng commented on HIVE-12441: -- [~alangates] Can you take a look as Eugene is not available? > Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed > -- > > Key: HIVE-12441 > URL: https://issues.apache.org/jira/browse/HIVE-12441 > Project: Hive > Issue Type: Bug > Components: CLI, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12441.1.patch, HIVE-12441.2.patch > > > recordValidTxns() is only needed if ACID tables are part of the query. > Otherwise it's just overhead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12441) Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed
[ https://issues.apache.org/jira/browse/HIVE-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139804#comment-15139804 ] Alan Gates commented on HIVE-12441: --- The patch looks fine, but I'm unclear what issue this is causing. Are we just saving an unnecessary call to the metastore? > Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed > -- > > Key: HIVE-12441 > URL: https://issues.apache.org/jira/browse/HIVE-12441 > Project: Hive > Issue Type: Bug > Components: CLI, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12441.1.patch, HIVE-12441.2.patch > > > recordValidTxns() is only needed if ACID tables are part of the query. > Otherwise it's just overhead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13029) NVDIMM support for LLAP Cache
[ https://issues.apache.org/jira/browse/HIVE-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139877#comment-15139877 ] Sergey Shelukhin commented on HIVE-13029: - Design doc? Or patch? ;) > NVDIMM support for LLAP Cache > - > > Key: HIVE-13029 > URL: https://issues.apache.org/jira/browse/HIVE-13029 > Project: Hive > Issue Type: New Feature > Components: llap >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Gopal V > > LLAP cache has been designed so that the cache can be offloaded easily to a > pmem API without restart coherence. > The tricky part about NVDIMMs are restart coherence, while most of the cache > gains can be obtained without keeping state across refreshes, since LLAP is > not the system of record, HDFS is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9862) Vectorized execution corrupts timestamp values
[
https://issues.apache.org/jira/browse/HIVE-9862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-9862:
---
Fix Version/s: 2.1.0
> Vectorized execution corrupts timestamp values
> --
>
> Key: HIVE-9862
> URL: https://issues.apache.org/jira/browse/HIVE-9862
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 1.0.0
>Reporter: Nathan Howell
>Assignee: Matt McCline
> Fix For: 2.1.0
>
> Attachments: HIVE-9862.01.patch, HIVE-9862.02.patch,
> HIVE-9862.03.patch, HIVE-9862.04.patch, HIVE-9862.05.patch,
> HIVE-9862.06.patch, HIVE-9862.07.patch, HIVE-9862.08.patch, HIVE-9862.09.patch
>
>
> Timestamps in the future (year 2250?) and before ~1700 are silently corrupted
> in vectorized execution mode. Simple repro:
> {code}
> hive> DROP TABLE IF EXISTS test;
> hive> CREATE TABLE test(ts TIMESTAMP) STORED AS ORC;
> hive> INSERT INTO TABLE test VALUES ('-12-31 23:59:59');
> hive> SET hive.vectorized.execution.enabled = false;
> hive> SELECT MAX(ts) FROM test;
> -12-31 23:59:59
> hive> SET hive.vectorized.execution.enabled = true;
> hive> SELECT MAX(ts) FROM test;
> 1816-03-30 05:56:07.066277376
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12892) Add global change versioning to permanent functions in metastore
[ https://issues.apache.org/jira/browse/HIVE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139452#comment-15139452 ] Sergey Shelukhin commented on HIVE-12892: - The version number for the set of functions (one that someone can query, or maybe get from ZK or whatever, to know whether fns have changed since they have last localized them). Dropping a function changes that number. > Add global change versioning to permanent functions in metastore > > > Key: HIVE-12892 > URL: https://issues.apache.org/jira/browse/HIVE-12892 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12892.01.patch, HIVE-12892.02.patch, > HIVE-12892.03.patch, HIVE-12892.04.patch, HIVE-12892.05.nogen.patch, > HIVE-12892.05.patch, HIVE-12892.05.patch, HIVE-12892.nogen.patch, > HIVE-12892.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13016) ORC FileDump recovery utility fails in Windows
[ https://issues.apache.org/jira/browse/HIVE-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139491#comment-15139491 ] Jason Dere commented on HIVE-13016: --- +1 > ORC FileDump recovery utility fails in Windows > -- > > Key: HIVE-13016 > URL: https://issues.apache.org/jira/browse/HIVE-13016 > Project: Hive > Issue Type: Bug >Affects Versions: 1.3.0, 2.1.0 >Reporter: Jason Dere >Assignee: Prasanth Jayachandran > Attachments: HIVE-13016.1.patch, HIVE-13016.2.patch > > > org.apache.hive.hcatalog.streaming.TestStreaming.testFileDumpCorruptDataFiles > org.apache.hive.hcatalog.streaming.TestStreaming.testFileDumpCorruptSideFiles > java.io.IOException: Unable to move > file:/E:/hive/hcatalog/streaming/target/tmp/junit4129594478393496260/testing3.db/dimensionTable/delta_001_002/bucket_0 > to > E:/hive/hcatalog/streaming/target/tmp/E:/hive/hcatalog/streaming/target/tmp/junit4129594478393496260/testing3.db/dimensionTable/delta_001_002/bucket_0 > at > org.apache.hadoop.hive.ql.io.orc.FileDump.moveFiles(FileDump.java:546)^M > at > org.apache.hadoop.hive.ql.io.orc.FileDump.recoverFile(FileDump.java:513)^M > at > org.apache.hadoop.hive.ql.io.orc.FileDump.recoverFiles(FileDump.java:428)^M > at org.apache.hadoop.hive.ql.io.orc.FileDump.main(FileDump.java:125)^M > at > org.apache.hive.hcatalog.streaming.TestStreaming.testFileDumpCorruptSideFiles(TestStreaming.java:1523)^M > Note that FileDump appends the full source path to the backup path when > trying to recover files (see "E:" in the middle of the destination path). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-1608) use sequencefile as the default for storing intermediate results
[ https://issues.apache.org/jira/browse/HIVE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang reassigned HIVE-1608: - Assignee: Chaoyu Tang (was: Brock Noland) > use sequencefile as the default for storing intermediate results > > > Key: HIVE-1608 > URL: https://issues.apache.org/jira/browse/HIVE-1608 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Namit Jain >Assignee: Chaoyu Tang > Attachments: HIVE-1608.1.patch, HIVE-1608.2.patch, HIVE-1608.3.patch, > HIVE-1608.4.patch, HIVE-1608.5.patch, HIVE-1608.patch > > > The only argument for having a text file for storing intermediate results > seems to be better debuggability. > But, tailing a sequence file is possible, and it should be more space > efficient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate
[
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139661#comment-15139661
]
Aihua Xu commented on HIVE-9534:
Attached patch-4: fix the unit test invalid_avg_syntax issue.
> incorrect result set for query that projects a windowed aggregate
> -
>
> Key: HIVE-9534
> URL: https://issues.apache.org/jira/browse/HIVE-9534
> Project: Hive
> Issue Type: Bug
> Components: PTF-Windowing
>Reporter: N Campbell
>Assignee: Aihua Xu
> Attachments: HIVE-9534.1.patch, HIVE-9534.2.patch, HIVE-9534.3.patch,
> HIVE-9534.4.patch
>
>
> Result set returned by Hive has one row instead of 5
> {code}
> select avg(distinct tsint.csint) over () from tsint
> create table if not exists TSINT (RNUM int , CSINT smallint)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
> STORED AS TEXTFILE;
> 0|\N
> 1|-1
> 2|0
> 3|1
> 4|10
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12941) Unexpected result when using MIN() on struct with NULL in first field
[
https://issues.apache.org/jira/browse/HIVE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-12941:
Attachment: (was: HIVE-12941.3.patch)
> Unexpected result when using MIN() on struct with NULL in first field
> -
>
> Key: HIVE-12941
> URL: https://issues.apache.org/jira/browse/HIVE-12941
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Jan-Erik Hedbom
>Assignee: Yongzhi Chen
> Attachments: HIVE-12941.1.patch, HIVE-12941.2.patch
>
>
> Using MIN() on struct with NULL in first field of a row yields NULL as result.
> Example:
> select min(a) FROM (select 1 as a union all select 2 as a union all select
> cast(null as int) as a) tmp;
> OK
> _c0
> 1
> As expected. But if we wrap it in a struct:
> select min(a) FROM (select named_struct("field",1) as a union all select
> named_struct("field",2) as a union all select named_struct("field",cast(null
> as int)) as a) tmp;
> OK
> _c0
> NULL
> Using MAX() works as expected for structs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12892) Add global change versioning to permanent functions in metastore
[ https://issues.apache.org/jira/browse/HIVE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139416#comment-15139416 ] Alan Gates commented on HIVE-12892: --- In the HBase side rather than create a whole new table you should use the existing sequences tables. I know this is completely screwed up at the moment since it's not one sequence per row as it should be, but I'll fix that as part of HIVE-12927. If you want to commit this as is and then I can fix it with HIVE-12927 that's fine too. Why do you increment the version number in drop function? It will be incremented again on add, so incrementing on drop seems unnecessary. The rest looks fine. > Add global change versioning to permanent functions in metastore > > > Key: HIVE-12892 > URL: https://issues.apache.org/jira/browse/HIVE-12892 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12892.01.patch, HIVE-12892.02.patch, > HIVE-12892.03.patch, HIVE-12892.04.patch, HIVE-12892.05.nogen.patch, > HIVE-12892.05.patch, HIVE-12892.05.patch, HIVE-12892.nogen.patch, > HIVE-12892.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7443) Fix HiveConnection to communicate with Kerberized Hive JDBC server and alternative JDKs
[ https://issues.apache.org/jira/browse/HIVE-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139638#comment-15139638 ] Greg Senia commented on HIVE-7443: -- [~crystal_gaoyu] did this fix ever make it into Hive? If it didn't by applying https://issues.apache.org/jira/browse/HADOOP-9969 this issue with beeline is gone with Hive 1.2.0 and the following fixes... https://issues.apache.org/jira/browse/TEZ-3105, https://issues.apache.org/jira/browse/HIVE-13020 > Fix HiveConnection to communicate with Kerberized Hive JDBC server and > alternative JDKs > --- > > Key: HIVE-7443 > URL: https://issues.apache.org/jira/browse/HIVE-7443 > Project: Hive > Issue Type: Bug > Components: JDBC, Security >Affects Versions: 0.12.0, 0.13.1 > Environment: Kerberos > Run Hive server2 and client with IBM JDK7.1 >Reporter: Yu Gao >Assignee: Yu Gao > Attachments: HIVE-7443.patch > > > Hive Kerberos authentication has been enabled in my cluster. I ran kinit to > initialize the current login user's ticket cache successfully, and then tried > to use beeline to connect to Hive Server2, but failed. After I manually added > some logging to catch the failure exception, this is what I got that caused > the failure: > beeline> !connect > jdbc:hive2://:1/default;principal=hive/@REALM.COM > org.apache.hive.jdbc.HiveDriver > scan complete in 2ms > Connecting to > jdbc:hive2://:1/default;principal=hive/@REALM.COM > Enter password for > jdbc:hive2://:1/default;principal=hive/@REALM.COM: > 14/07/17 15:12:45 ERROR jdbc.HiveConnection: Failed to open client transport > javax.security.sasl.SaslException: Failed to open client transport [Caused by > java.io.IOException: Could not instantiate SASL transport] > at > org.apache.hive.service.auth.KerberosSaslHelper.getKerberosTransport(KerberosSaslHelper.java:78) > at > org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:342) > at > org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:200) > at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:178) > at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) > at java.sql.DriverManager.getConnection(DriverManager.java:582) > at java.sql.DriverManager.getConnection(DriverManager.java:198) > at > org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145) > at > org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:186) > at org.apache.hive.beeline.Commands.connect(Commands.java:959) > at org.apache.hive.beeline.Commands.connect(Commands.java:880) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) > at java.lang.reflect.Method.invoke(Method.java:619) > at > org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:44) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:801) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:659) > at > org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:368) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:351) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) > at java.lang.reflect.Method.invoke(Method.java:619) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > Caused by: java.io.IOException: Could not instantiate SASL transport > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Client.createClientTransport(HadoopThriftAuthBridge20S.java:177) > at > org.apache.hive.service.auth.KerberosSaslHelper.getKerberosTransport(KerberosSaslHelper.java:74) > ... 24 more > Caused by: javax.security.sasl.SaslException: Failure to initialize security > context [Caused by org.ietf.jgss.GSSException, major code: 13, minor code: 0 > major string: Invalid credentials > minor string: SubjectCredFinder: no JAAS Subject] > at > com.ibm.security.sasl.gsskerb.GssKrb5Client.(GssKrb5Client.java:131) > at > com.ibm.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:53) > at javax.security.sasl.Sasl.createSaslClient(Sasl.java:362) > at >
[jira] [Commented] (HIVE-12987) Add metrics for HS2 active users and SQL operations
[ https://issues.apache.org/jira/browse/HIVE-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139315#comment-15139315 ] Jimmy Xiang commented on HIVE-12987: Thanks a lot for the review. Yeah we need to wait for the new build. The tests failure above are not related. > Add metrics for HS2 active users and SQL operations > --- > > Key: HIVE-12987 > URL: https://issues.apache.org/jira/browse/HIVE-12987 > Project: Hive > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-12987.1.patch, HIVE-12987.2.patch, > HIVE-12987.2.patch, HIVE-12987.3.patch, HIVE-12987.3.patch, HIVE-12987.4.patch > > > HIVE-12271 added metrics for all HS2 operations. Sometimes, users are also > interested in metrics just for SQL operations. > It is useful to track active user count as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12941) Unexpected result when using MIN() on struct with NULL in first field
[
https://issues.apache.org/jira/browse/HIVE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-12941:
Attachment: HIVE-12941.3.patch
Attach patch 3 after review to fix function name related issues
> Unexpected result when using MIN() on struct with NULL in first field
> -
>
> Key: HIVE-12941
> URL: https://issues.apache.org/jira/browse/HIVE-12941
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Jan-Erik Hedbom
>Assignee: Yongzhi Chen
> Attachments: HIVE-12941.1.patch, HIVE-12941.2.patch,
> HIVE-12941.3.patch
>
>
> Using MIN() on struct with NULL in first field of a row yields NULL as result.
> Example:
> select min(a) FROM (select 1 as a union all select 2 as a union all select
> cast(null as int) as a) tmp;
> OK
> _c0
> 1
> As expected. But if we wrap it in a struct:
> select min(a) FROM (select named_struct("field",1) as a union all select
> named_struct("field",2) as a union all select named_struct("field",cast(null
> as int)) as a) tmp;
> OK
> _c0
> NULL
> Using MAX() works as expected for structs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12941) Unexpected result when using MIN() on struct with NULL in first field
[
https://issues.apache.org/jira/browse/HIVE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139363#comment-15139363
]
Aihua Xu commented on HIVE-12941:
-
The patch looks good. Nit:a comment needs to be updated {{if nullAsMax true,
treat null as maximum value.}} and {{compare two object => two objects}}.
+1.
> Unexpected result when using MIN() on struct with NULL in first field
> -
>
> Key: HIVE-12941
> URL: https://issues.apache.org/jira/browse/HIVE-12941
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Jan-Erik Hedbom
>Assignee: Yongzhi Chen
> Attachments: HIVE-12941.1.patch, HIVE-12941.2.patch,
> HIVE-12941.3.patch
>
>
> Using MIN() on struct with NULL in first field of a row yields NULL as result.
> Example:
> select min(a) FROM (select 1 as a union all select 2 as a union all select
> cast(null as int) as a) tmp;
> OK
> _c0
> 1
> As expected. But if we wrap it in a struct:
> select min(a) FROM (select named_struct("field",1) as a union all select
> named_struct("field",2) as a union all select named_struct("field",cast(null
> as int)) as a) tmp;
> OK
> _c0
> NULL
> Using MAX() works as expected for structs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10632) Make sure TXN_COMPONENTS gets cleaned up if table is dropped before compaction.
[ https://issues.apache.org/jira/browse/HIVE-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zheng updated HIVE-10632: - Attachment: HIVE-10632.3.patch patch 3 for test > Make sure TXN_COMPONENTS gets cleaned up if table is dropped before > compaction. > --- > > Key: HIVE-10632 > URL: https://issues.apache.org/jira/browse/HIVE-10632 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng >Priority: Critical > Attachments: HIVE-10632.1.patch, HIVE-10632.2.patch, > HIVE-10632.3.patch > > > The compaction process will clean up entries in TXNS, > COMPLETED_TXN_COMPONENTS, TXN_COMPONENTS. If the table/partition is dropped > before compaction is complete there will be data left in these tables. Need > to investigate if there are other situations where this may happen and > address it. > see HIVE-10595 for additional info -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12441) Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed
[ https://issues.apache.org/jira/browse/HIVE-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zheng updated HIVE-12441: - Attachment: HIVE-12441.branch-1.patch > Driver.acquireLocksAndOpenTxn() should only call recordValidTxns() when needed > -- > > Key: HIVE-12441 > URL: https://issues.apache.org/jira/browse/HIVE-12441 > Project: Hive > Issue Type: Bug > Components: CLI, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12441.1.patch, HIVE-12441.2.patch, > HIVE-12441.branch-1.patch > > > recordValidTxns() is only needed if ACID tables are part of the query. > Otherwise it's just overhead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12558) LLAP: output QueryFragmentCounters somewhere
[ https://issues.apache.org/jira/browse/HIVE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140126#comment-15140126 ] Prasanth Jayachandran commented on HIVE-12558: -- Addressed code review comments > LLAP: output QueryFragmentCounters somewhere > > > Key: HIVE-12558 > URL: https://issues.apache.org/jira/browse/HIVE-12558 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Sergey Shelukhin >Assignee: Prasanth Jayachandran > Attachments: HIVE-12558.1.patch, HIVE-12558.2.patch, > HIVE-12558.wip.patch, sample-output.png > > > Right now, LLAP logs counters for every fragment; most of them are IO related > and could be very useful, they also include table names so that things like > cache hit ratio, etc., could be calculated for every table. > We need to output them to some metrics system (preserving the breakdown by > table, possibly also adding query ID or even stage) so that they'd be usable > without grep/sed/awk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12918) LLAP should never create embedded metastore when localizing functions
[
https://issues.apache.org/jira/browse/HIVE-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139955#comment-15139955
]
Sergey Shelukhin commented on HIVE-12918:
-
[~ashutoshc] [~jdere] can you review?
> LLAP should never create embedded metastore when localizing functions
> -
>
> Key: HIVE-12918
> URL: https://issues.apache.org/jira/browse/HIVE-12918
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.1.0
>Reporter: Siddharth Seth
>Assignee: Sergey Shelukhin
> Attachments: HIVE-12918.01.patch, HIVE-12918.02.patch,
> HIVE-12918.03.patch, HIVE-12918.patch
>
>
> {code}
> 16/01/24 21:29:02 INFO service.AbstractService: Service LlapDaemon failed in
> state INITED; cause: java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1552)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3110)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3130)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3355)
> at
> org.apache.hadoop.hive.llap.daemon.impl.FunctionLocalizer.startLocalizeAllFunctions(FunctionLocalizer.java:88)
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceInit(LlapDaemon.java:244)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.main(LlapDaemon.java:323)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1550)
> ... 10 more
> Caused by: java.lang.NoClassDefFoundError: org/datanucleus/NucleusContext
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getClass(MetaStoreUtils.java:1517)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:61)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:568)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:533)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:595)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:387)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:78)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5935)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:221)
> at
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:67)
> ... 15 more
> Caused by: java.lang.ClassNotFoundException: org.datanucleus.NucleusContext
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> {code}
> Looks like the DataNucleus jar is not added to the LLAP classpath. This
> appears to be caused by HIVE-12853
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10115) HS2 running on a Kerberized cluster should offer Kerberos(GSSAPI) and Delegation token(DIGEST) when alternate authentication is enabled
[ https://issues.apache.org/jira/browse/HIVE-10115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139969#comment-15139969 ] Lefty Leverenz commented on HIVE-10115: --- Okay, thanks Sergio. > HS2 running on a Kerberized cluster should offer Kerberos(GSSAPI) and > Delegation token(DIGEST) when alternate authentication is enabled > --- > > Key: HIVE-10115 > URL: https://issues.apache.org/jira/browse/HIVE-10115 > Project: Hive > Issue Type: Improvement > Components: Authentication >Affects Versions: 1.1.0 >Reporter: Mubashir Kazia >Assignee: Mubashir Kazia > Labels: patch > Fix For: 1.3.0, 2.1.0 > > Attachments: HIVE-10115.0.patch, HIVE-10115.2.patch > > > In a Kerberized cluster when alternate authentication is enabled on HS2, it > should also accept Kerberos Authentication. The reason this is important is > because when we enable LDAP authentication HS2 stops accepting delegation > token authentication. So we are forced to enter username passwords in the > oozie configuration. > The whole idea of SASL is that multiple authentication mechanism can be > offered. If we disable Kerberos(GSSAPI) and delegation token (DIGEST) > authentication when we enable LDAP authentication, this defeats SASL purpose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13029) NVDIMM support for LLAP Cache
[ https://issues.apache.org/jira/browse/HIVE-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139988#comment-15139988 ] Gopal V commented on HIVE-13029: Patch :) > NVDIMM support for LLAP Cache > - > > Key: HIVE-13029 > URL: https://issues.apache.org/jira/browse/HIVE-13029 > Project: Hive > Issue Type: New Feature > Components: llap >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Gopal V > > LLAP cache has been designed so that the cache can be offloaded easily to a > pmem API without restart coherence. > The tricky part about NVDIMMs are restart coherence, while most of the cache > gains can be obtained without keeping state across refreshes, since LLAP is > not the system of record, HDFS is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12558) LLAP: output QueryFragmentCounters somewhere
[ https://issues.apache.org/jira/browse/HIVE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-12558: - Attachment: HIVE-12558.2.patch > LLAP: output QueryFragmentCounters somewhere > > > Key: HIVE-12558 > URL: https://issues.apache.org/jira/browse/HIVE-12558 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Sergey Shelukhin >Assignee: Prasanth Jayachandran > Attachments: HIVE-12558.1.patch, HIVE-12558.2.patch, > HIVE-12558.wip.patch, sample-output.png > > > Right now, LLAP logs counters for every fragment; most of them are IO related > and could be very useful, they also include table names so that things like > cache hit ratio, etc., could be calculated for every table. > We need to output them to some metrics system (preserving the breakdown by > table, possibly also adding query ID or even stage) so that they'd be usable > without grep/sed/awk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12924) CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_ppr_multi_distinct.q failure
[
https://issues.apache.org/jira/browse/HIVE-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140051#comment-15140051
]
Hari Sankar Sivarama Subramaniyan commented on HIVE-12924:
--
The failures are unrelated to the change. cc-ing [~jpullokkaran] for review
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver
> groupby_ppr_multi_distinct.q failure
>
>
> Key: HIVE-12924
> URL: https://issues.apache.org/jira/browse/HIVE-12924
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-12924.1.patch, HIVE-12924.2.patch,
> HIVE-12924.3.patch
>
>
> {code}
> EXPLAIN EXTENDED
> FROM srcpart src
> INSERT OVERWRITE TABLE dest1
> SELECT substr(src.key,1,1), count(DISTINCT substr(src.value,5)),
> concat(substr(src.key,1,1),sum(substr(src.value,5))), sum(DISTINCT
> substr(src.value, 5)), count(DISTINCT src.value)
> WHERE src.ds = '2008-04-08'
> GROUP BY substr(src.key,1,1)
> {code}
> Ended Job = job_local968043618_0742 with errors
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12995) LLAP: Synthetic file ids need collision checks
[ https://issues.apache.org/jira/browse/HIVE-12995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140149#comment-15140149 ] Sergey Shelukhin commented on HIVE-12995: - Not working on it yet, just assigned so it would show up in my filters... > LLAP: Synthetic file ids need collision checks > -- > > Key: HIVE-12995 > URL: https://issues.apache.org/jira/browse/HIVE-12995 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Sergey Shelukhin > > LLAP synthetic file ids do not have any way of checking whether a collision > occurs other than a data-error. > Synthetic file-ids have only been used with unit tests so far - but they will > be needed to add cache mechanisms to non-HDFS filesystems. > In case of Synthetic file-ids, it is recommended that we track the full-tuple > (path, mtime, len) in the cache so that a cache-hit for the synthetic file-id > can be compared against the parameters & only accepted if those match. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-12995) LLAP: Synthetic file ids need collision checks
[ https://issues.apache.org/jira/browse/HIVE-12995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-12995: --- Assignee: Sergey Shelukhin > LLAP: Synthetic file ids need collision checks > -- > > Key: HIVE-12995 > URL: https://issues.apache.org/jira/browse/HIVE-12995 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Sergey Shelukhin > > LLAP synthetic file ids do not have any way of checking whether a collision > occurs other than a data-error. > Synthetic file-ids have only been used with unit tests so far - but they will > be needed to add cache mechanisms to non-HDFS filesystems. > In case of Synthetic file-ids, it is recommended that we track the full-tuple > (path, mtime, len) in the cache so that a cache-hit for the synthetic file-id > can be compared against the parameters & only accepted if those match. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12924) CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_ppr_multi_distinct.q failure
[
https://issues.apache.org/jira/browse/HIVE-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140035#comment-15140035
]
Hive QA commented on HIVE-12924:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786962/HIVE-12924.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9768 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6925/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6925/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6925/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786962 - PreCommit-HIVE-TRUNK-Build
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver
> groupby_ppr_multi_distinct.q failure
>
>
> Key: HIVE-12924
> URL: https://issues.apache.org/jira/browse/HIVE-12924
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-12924.1.patch, HIVE-12924.2.patch,
> HIVE-12924.3.patch
>
>
> {code}
> EXPLAIN EXTENDED
> FROM srcpart src
> INSERT OVERWRITE TABLE dest1
> SELECT substr(src.key,1,1), count(DISTINCT substr(src.value,5)),
> concat(substr(src.key,1,1),sum(substr(src.value,5))), sum(DISTINCT
> substr(src.value, 5)), count(DISTINCT src.value)
> WHERE src.ds = '2008-04-08'
> GROUP BY substr(src.key,1,1)
> {code}
> Ended Job = job_local968043618_0742 with errors
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12558) LLAP: output QueryFragmentCounters somewhere
[ https://issues.apache.org/jira/browse/HIVE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-12558: - Attachment: (was: HIVE-12558.2.patch) > LLAP: output QueryFragmentCounters somewhere > > > Key: HIVE-12558 > URL: https://issues.apache.org/jira/browse/HIVE-12558 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Sergey Shelukhin >Assignee: Prasanth Jayachandran > Attachments: HIVE-12558.1.patch, HIVE-12558.2.patch, > HIVE-12558.wip.patch, sample-output.png > > > Right now, LLAP logs counters for every fragment; most of them are IO related > and could be very useful, they also include table names so that things like > cache hit ratio, etc., could be calculated for every table. > We need to output them to some metrics system (preserving the breakdown by > table, possibly also adding query ID or even stage) so that they'd be usable > without grep/sed/awk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12558) LLAP: output QueryFragmentCounters somewhere
[ https://issues.apache.org/jira/browse/HIVE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-12558: - Attachment: HIVE-12558.2.patch > LLAP: output QueryFragmentCounters somewhere > > > Key: HIVE-12558 > URL: https://issues.apache.org/jira/browse/HIVE-12558 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Sergey Shelukhin >Assignee: Prasanth Jayachandran > Attachments: HIVE-12558.1.patch, HIVE-12558.2.patch, > HIVE-12558.wip.patch, sample-output.png > > > Right now, LLAP logs counters for every fragment; most of them are IO related > and could be very useful, they also include table names so that things like > cache hit ratio, etc., could be calculated for every table. > We need to output them to some metrics system (preserving the breakdown by > table, possibly also adding query ID or even stage) so that they'd be usable > without grep/sed/awk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13032) Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization
[ https://issues.apache.org/jira/browse/HIVE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140144#comment-15140144 ] Sergey Shelukhin commented on HIVE-13032: - +1. Can you see that no other services have this line explicitly? Will this cause problems for CLI due to both inits? > Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization > -- > > Key: HIVE-13032 > URL: https://issues.apache.org/jira/browse/HIVE-13032 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Blocker > Attachments: HIVE-13032.1.patch, HIVE-13032.2.patch > > > HIVE-12497 removed HADOOP_CLIENT_OPTS as it slowed down cli launch time. But > it leads to log4j2 not being initialized when using services other than CLI. > Other services like metastore, schematool etc. rely on log4j to initialize > the logging based on the presence of log4j2.properties file in the classpath. > If we use the standard name for log4j configuration file (log4j2.properties) > then automatic initialization will happen. If not, we have to tell log4j to > look for specific properties file. This is done via -Dlog4j.configurationFile > system property. If we pass this system property via HADOOP_CLIENT_OPTS then > all hive services will have logging initialized properly. > In HIVE-12497, the problem was we had HADOOP_CLIENT_OPTS at the top of the > script. As a result, hadoop and hbase commands tries to initialize logging > which took long time slowing down the startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-13028) Remove javadoc plugin from webhcat
[ https://issues.apache.org/jira/browse/HIVE-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-13028: --- Assignee: Sergey Shelukhin > Remove javadoc plugin from webhcat > -- > > Key: HIVE-13028 > URL: https://issues.apache.org/jira/browse/HIVE-13028 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13028.patch > > > Webhcat has about 3 million javadoc errors that nobody, presumably, cares > about. It also has its very own javadoc section in the pom that causes mvn > deploy to fail with 3 million javadoc errors, even when maven.javadoc.skip is > true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13028) Remove javadoc plugin from webhcat
[ https://issues.apache.org/jira/browse/HIVE-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13028: Attachment: HIVE-13028.patch [~ekoifman] [~sushanth] can you take a look? > Remove javadoc plugin from webhcat > -- > > Key: HIVE-13028 > URL: https://issues.apache.org/jira/browse/HIVE-13028 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13028.patch > > > Webhcat has about 3 million javadoc errors that nobody, presumably, cares > about. It also has its very own javadoc section in the pom that causes mvn > deploy to fail with 3 million javadoc errors, even when maven.javadoc.skip is > true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11355) Hive on tez: memory manager for sort buffers (input/output) and operators
[ https://issues.apache.org/jira/browse/HIVE-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-11355: -- Attachment: HIVE-11355.14.patch Rebased. [~gopalv] Could you take a look please? > Hive on tez: memory manager for sort buffers (input/output) and operators > - > > Key: HIVE-11355 > URL: https://issues.apache.org/jira/browse/HIVE-11355 > Project: Hive > Issue Type: Improvement > Components: Tez >Affects Versions: 2.0.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-11355.1.patch, HIVE-11355.10.patch, > HIVE-11355.11.patch, HIVE-11355.12.patch, HIVE-11355.13.patch, > HIVE-11355.14.patch, HIVE-11355.2.patch, HIVE-11355.3.patch, > HIVE-11355.4.patch, HIVE-11355.5.patch, HIVE-11355.6.patch, > HIVE-11355.7.patch, HIVE-11355.8.patch, HIVE-11355.9.patch > > > We need to better manage the sort buffer allocations to ensure better > performance. Also, we need to provide configurations to certain operators to > stay within memory limits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13032) Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization
[ https://issues.apache.org/jira/browse/HIVE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-13032: - Priority: Blocker (was: Major) > Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization > -- > > Key: HIVE-13032 > URL: https://issues.apache.org/jira/browse/HIVE-13032 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Blocker > Attachments: HIVE-13032.1.patch > > > HIVE-12497 removed HADOOP_CLIENT_OPTS as it slowed down cli launch time. But > it leads to log4j2 not being initialized when using services other than CLI. > Other services like metastore, schematool etc. rely on log4j to initialize > the logging based on the presence of log4j2.properties file in the classpath. > If we use the standard name for log4j configuration file (log4j2.properties) > then automatic initialization will happen. If not, we have to tell log4j to > look for specific properties file. This is done via -Dlog4j.configurationFile > system property. If we pass this system property via HADOOP_CLIENT_OPTS then > all hive services will have logging initialized properly. > In HIVE-12497, the problem was we had HADOOP_CLIENT_OPTS at the top of the > script. As a result, hadoop and hbase commands tries to initialize logging > which took long time slowing down the startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13032) Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization
[ https://issues.apache.org/jira/browse/HIVE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140139#comment-15140139 ] Prasanth Jayachandran commented on HIVE-13032: -- [~sershe] Can you please take a look? > Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization > -- > > Key: HIVE-13032 > URL: https://issues.apache.org/jira/browse/HIVE-13032 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Blocker > Attachments: HIVE-13032.1.patch, HIVE-13032.2.patch > > > HIVE-12497 removed HADOOP_CLIENT_OPTS as it slowed down cli launch time. But > it leads to log4j2 not being initialized when using services other than CLI. > Other services like metastore, schematool etc. rely on log4j to initialize > the logging based on the presence of log4j2.properties file in the classpath. > If we use the standard name for log4j configuration file (log4j2.properties) > then automatic initialization will happen. If not, we have to tell log4j to > look for specific properties file. This is done via -Dlog4j.configurationFile > system property. If we pass this system property via HADOOP_CLIENT_OPTS then > all hive services will have logging initialized properly. > In HIVE-12497, the problem was we had HADOOP_CLIENT_OPTS at the top of the > script. As a result, hadoop and hbase commands tries to initialize logging > which took long time slowing down the startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12984) spark tgz-s need to be deleted on mvn clean, as are other binary artifacts in the tree
[ https://issues.apache.org/jira/browse/HIVE-12984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140213#comment-15140213 ] Sergey Shelukhin commented on HIVE-12984: - They actually get pulled into deployed artifacts on mvn deploy, I don't think out packaging filters can affect that. I am cutting the 4th RC now and it's kind of tiresome to delete them every time. Why do spark tests need to depend on the package instead of just individual jars? MiniTez and MiniMr (and MiniHdfs and MiniHBase and MiniAccumulo) all run fine with just jars. Otherwise, is it possible to file a spark JIRA to have a distribution without Hive if there's no way around it? > spark tgz-s need to be deleted on mvn clean, as are other binary artifacts in > the tree > -- > > Key: HIVE-12984 > URL: https://issues.apache.org/jira/browse/HIVE-12984 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12984.patch > > > Currently, tgz files are downloaded and kept around forever. I noticed when > packaging the release (apparently the excludes in packaging files also didn't > work) that the initial src tar.gz was huge; regardless of that, I had 6 > version of spark (1.2 thru 1.6 with one dot version) sitting there, and also > in every clone of Hive that I have. > These should be switched to use normal means of artifact distribution (I > think I already filed a jira but I cannot find it now); meanwhile making sure > that mvn clean would remove them. > I realize it could create some pain when running tests repeatedly on dev > machine unless "clean" is omitted from rebuilds; that is somewhat intentional > - it should be a good incentive to switch to maven for dependency management > instead of a bash script ;) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13032) Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization
[ https://issues.apache.org/jira/browse/HIVE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140196#comment-15140196 ] Prasanth Jayachandran commented on HIVE-13032: -- Verified that it's not included in other services. Beeline uses different properties file and sets log4j.configurationFile again. Also double initialization is not a problem as logger factory is initialized in a static block which will be initialized automatically by the class loader. Using Configurator to initialize again will return the same instance. Committing this patch shortly. > Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization > -- > > Key: HIVE-13032 > URL: https://issues.apache.org/jira/browse/HIVE-13032 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Blocker > Attachments: HIVE-13032.1.patch, HIVE-13032.2.patch > > > HIVE-12497 removed HADOOP_CLIENT_OPTS as it slowed down cli launch time. But > it leads to log4j2 not being initialized when using services other than CLI. > Other services like metastore, schematool etc. rely on log4j to initialize > the logging based on the presence of log4j2.properties file in the classpath. > If we use the standard name for log4j configuration file (log4j2.properties) > then automatic initialization will happen. If not, we have to tell log4j to > look for specific properties file. This is done via -Dlog4j.configurationFile > system property. If we pass this system property via HADOOP_CLIENT_OPTS then > all hive services will have logging initialized properly. > In HIVE-12497, the problem was we had HADOOP_CLIENT_OPTS at the top of the > script. As a result, hadoop and hbase commands tries to initialize logging > which took long time slowing down the startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10187) Avro backed tables don't handle cyclical or recursive records
[ https://issues.apache.org/jira/browse/HIVE-10187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140210#comment-15140210 ] Carl Steinbach commented on HIVE-10187: --- +1. Will commit if the test results come back clean. > Avro backed tables don't handle cyclical or recursive records > - > > Key: HIVE-10187 > URL: https://issues.apache.org/jira/browse/HIVE-10187 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 1.2.0 >Reporter: Mark Wagner >Assignee: Mark Wagner > Attachments: HIVE-10187.1.patch, HIVE-10187.2.patch, > HIVE-10187.3.patch, HIVE-10187.4.patch, HIVE-10187.5.patch, > HIVE-10187.demo.patch > > > [HIVE-7653] changed the Avro SerDe to make it generate TypeInfos even for > recursive/cyclical schemas. However, any attempt to serialize data which > exploits that ability results in silently dropped fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11866) Add framework to enable testing using LDAPServer using LDAP protocol
[
https://issues.apache.org/jira/browse/HIVE-11866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140288#comment-15140288
]
Hive QA commented on HIVE-11866:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786978/HIVE-11866.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9773 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6927/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6927/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6927/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786978 - PreCommit-HIVE-TRUNK-Build
> Add framework to enable testing using LDAPServer using LDAP protocol
>
>
> Key: HIVE-11866
> URL: https://issues.apache.org/jira/browse/HIVE-11866
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.3.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-11866.2.patch, HIVE-11866.3.patch,
> HIVE-11866.4.patch, HIVE-11866.patch
>
>
> Currently there is no unit test coverage for HS2's LDAP Atn provider using a
> LDAP Server on the backend. This prevents testing of the LDAPAtnProvider with
> some realistic usecases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13033) SPDO unnecessarily duplicates columns in key & value of mapper output
[ https://issues.apache.org/jira/browse/HIVE-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-13033: Attachment: HIVE-13033.patch > SPDO unnecessarily duplicates columns in key & value of mapper output > - > > Key: HIVE-13033 > URL: https://issues.apache.org/jira/browse/HIVE-13033 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-13033.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13013) Further Improve concurrency in TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-13013: -- Attachment: HIVE-13013.patch [~alangates] could you review please > Further Improve concurrency in TxnHandler > - > > Key: HIVE-13013 > URL: https://issues.apache.org/jira/browse/HIVE-13013 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-13013.patch > > > There are still a few operations in TxnHandler that run at Serializable > isolation. > Most or all of them can be dropped to READ_COMMITTED now that we have SELECT > ... FOR UPDATE support. This will reduce number of deadlocks in the DBs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12237) Use slf4j as logging facade
[
https://issues.apache.org/jira/browse/HIVE-12237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140253#comment-15140253
]
Carter Shanklin commented on HIVE-12237:
The docs do need an update as I've found, when I follow the old style of hive
--service hiveserver2 --hiveconf hive.root.logger=DEBUG,console on a Hive 2
build I get
{code}
2016-02-09 22:20:58,104 main ERROR Unable to locate appender DEBUG,console for
logger
2016-02-09 22:20:58,106 main ERROR Unable to locate appender DEBUG,console for
logger JPOX.Query
2016-02-09 22:20:58,107 main ERROR Unable to locate appender DEBUG,console for
logger JPOX.General
2016-02-09 22:20:58,107 main ERROR Unable to locate appender DEBUG,console for
logger JPOX.Enhancer
2016-02-09 22:20:58,108 main ERROR Unable to locate appender DEBUG,console for
logger JPOX.Metadata
2016-02-09 22:20:58,108 main ERROR Unable to locate appender DEBUG,console for
logger JPOX.Datastore
2016-02-09 22:20:58,108 main ERROR Unable to locate appender DEBUG,console for
logger JPOX.Plugin
2016-02-09 22:20:58,109 main ERROR Unable to locate appender DEBUG,console for
logger DataNucleus
2016-02-09 22:20:58,109 main ERROR Unable to locate appender DEBUG,console for
logger Datastore
2016-02-09 22:20:58,110 main ERROR Unable to locate appender DEBUG,console for
logger org.apache.zookeeper.ClientCnxnSocketNIO
2016-02-09 22:20:58,110 main ERROR Unable to locate appender DEBUG,console for
logger org.apache.zookeeper.server.NIOServerCnxn
2016-02-09 22:20:58,111 main ERROR Unable to locate appender DEBUG,console for
logger Datastore.Schema
{code}
There was also this warning:
ERROR StatusLogger No log4j2 configuration file found. Using default
configuration: logging only errors to the console.
When I renamed the template file hive-log4j2.xml.template to hive-log4j2.xml
this error doesn't go away. I needed to create a new file named log4j2.xml
instead.
Even so, setting all the levels there to DEBUG I'm not seeing all the log
messages I used to see on Hive 1. I see a bunch of messages when HS2 starts but
then nothing.
So in any event it's worth a doc update, but it would be really unfortunate if
I couldn't control logging from the command line anymore.
> Use slf4j as logging facade
> ---
>
> Key: HIVE-12237
> URL: https://issues.apache.org/jira/browse/HIVE-12237
> Project: Hive
> Issue Type: Task
> Components: Logging
>Affects Versions: 2.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Fix For: 2.0.0
>
> Attachments: HIVE-12237.1.patch, HIVE-12237.2.patch,
> HIVE-12237.3.patch, HIVE-12237.4.patch, HIVE-12237.5.patch,
> HIVE-12237.6.patch, HIVE-12237.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13022) Header in output file
[ https://issues.apache.org/jira/browse/HIVE-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140293#comment-15140293 ] Minu commented on HIVE-13022: - To prevent the Hadoop table names from being prepended to the column names, try using this property setting in the Hive-site.xml file. hive.resultset.use.unique.column.names false > Header in output file > -- > > Key: HIVE-13022 > URL: https://issues.apache.org/jira/browse/HIVE-13022 > Project: Hive > Issue Type: Bug > Environment: Hdoop 2.1.0, hive >Reporter: raju puli > Original Estimate: 12h > Remaining Estimate: 12h > > When i'm printing my out put with the header, each field name is coming with > _u1. , i need fix this one as soon as possible my project is on production > stage it need to upload to the web, it will make lot of conflicts. > for example my field names are looks like this > field1 field2 field3 field4 > but in out put it's printing like this > _u1.field1 _u1.field2 _u1.field3 _u1.field4 > my script looks like this > select * from( > ) > union all > select * from ( > ); > i'm running this script with using TEZ. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-13032) Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization
[ https://issues.apache.org/jira/browse/HIVE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-13032. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.0 Committed to branch-2.0 and master > Hive services need HADOOP_CLIENT_OPTS for proper log4j2 initialization > -- > > Key: HIVE-13032 > URL: https://issues.apache.org/jira/browse/HIVE-13032 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Blocker > Fix For: 2.0.0, 2.1.0 > > Attachments: HIVE-13032.1.patch, HIVE-13032.2.patch > > > HIVE-12497 removed HADOOP_CLIENT_OPTS as it slowed down cli launch time. But > it leads to log4j2 not being initialized when using services other than CLI. > Other services like metastore, schematool etc. rely on log4j to initialize > the logging based on the presence of log4j2.properties file in the classpath. > If we use the standard name for log4j configuration file (log4j2.properties) > then automatic initialization will happen. If not, we have to tell log4j to > look for specific properties file. This is done via -Dlog4j.configurationFile > system property. If we pass this system property via HADOOP_CLIENT_OPTS then > all hive services will have logging initialized properly. > In HIVE-12497, the problem was we had HADOOP_CLIENT_OPTS at the top of the > script. As a result, hadoop and hbase commands tries to initialize logging > which took long time slowing down the startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-12984) spark tgz-s need to be deleted on mvn clean, as are other binary artifacts in the tree
[ https://issues.apache.org/jira/browse/HIVE-12984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140213#comment-15140213 ] Sergey Shelukhin edited comment on HIVE-12984 at 2/10/16 2:15 AM: -- They actually get pulled into deployed artifacts on mvn deploy, I don't think out packaging filters can affect that. I am cutting the 4th RC now and it's kind of tiresome to delete them every time (after forgetting and noticing the source zip is 300Mb). Why do spark tests need to depend on the package instead of just individual jars? MiniTez and MiniMr (and MiniHdfs and MiniHBase and MiniAccumulo) all run fine with just jars. Otherwise, is it possible to file a spark JIRA to have a distribution without Hive if there's no way around it? was (Author: sershe): They actually get pulled into deployed artifacts on mvn deploy, I don't think out packaging filters can affect that. I am cutting the 4th RC now and it's kind of tiresome to delete them every time. Why do spark tests need to depend on the package instead of just individual jars? MiniTez and MiniMr (and MiniHdfs and MiniHBase and MiniAccumulo) all run fine with just jars. Otherwise, is it possible to file a spark JIRA to have a distribution without Hive if there's no way around it? > spark tgz-s need to be deleted on mvn clean, as are other binary artifacts in > the tree > -- > > Key: HIVE-12984 > URL: https://issues.apache.org/jira/browse/HIVE-12984 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12984.patch > > > Currently, tgz files are downloaded and kept around forever. I noticed when > packaging the release (apparently the excludes in packaging files also didn't > work) that the initial src tar.gz was huge; regardless of that, I had 6 > version of spark (1.2 thru 1.6 with one dot version) sitting there, and also > in every clone of Hive that I have. > These should be switched to use normal means of artifact distribution (I > think I already filed a jira but I cannot find it now); meanwhile making sure > that mvn clean would remove them. > I realize it could create some pain when running tests repeatedly on dev > machine unless "clean" is omitted from rebuilds; that is somewhat intentional > - it should be a good incentive to switch to maven for dependency management > instead of a bash script ;) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13002) metastore call timing is not threadsafe
[
https://issues.apache.org/jira/browse/HIVE-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140237#comment-15140237
]
Hive QA commented on HIVE-13002:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12786963/HIVE-13002.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 9753 tests executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-script_pipe.q-tez_smb_empty.q-mapjoin_decimal.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarDataNucleusUnCaching
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6926/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6926/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6926/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12786963 - PreCommit-HIVE-TRUNK-Build
> metastore call timing is not threadsafe
> ---
>
> Key: HIVE-13002
> URL: https://issues.apache.org/jira/browse/HIVE-13002
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13002.01.patch, HIVE-13002.patch
>
>
> Discovered in some q test run:
> {noformat}
> TestCliDriver.testCliDriver_insert_values_orig_table:123->runTest:199
> Unexpected exception java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
> at java.util.HashMap$EntryIterator.next(HashMap.java:966)
> at java.util.HashMap$EntryIterator.next(HashMap.java:964)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.dumpAndClearMetaCallTiming(Hive.java:3412)
> at
> org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(Driver.java:574)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1722)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1342)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1113)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1101)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-13034) Add jdeb plugin to build debian
[ https://issues.apache.org/jira/browse/HIVE-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arshad Matin reassigned HIVE-13034: --- Assignee: Arshad Matin > Add jdeb plugin to build debian > --- > > Key: HIVE-13034 > URL: https://issues.apache.org/jira/browse/HIVE-13034 > Project: Hive > Issue Type: Improvement >Reporter: Arshad Matin >Assignee: Arshad Matin > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12960) Migrate Column Stats Extrapolation to HBaseStore
[ https://issues.apache.org/jira/browse/HIVE-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12960: --- Attachment: HIVE-12960.01.patch > Migrate Column Stats Extrapolation to HBaseStore > > > Key: HIVE-12960 > URL: https://issues.apache.org/jira/browse/HIVE-12960 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-12960.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12960) Migrate Column Stats Extrapolation to HBaseStore
[ https://issues.apache.org/jira/browse/HIVE-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140418#comment-15140418 ] Pengcheng Xiong commented on HIVE-12960: [~sershe], i just submitted the initial patch. I agree that we should (1) move the logic out of directsql and (2) it would be the best if ObjectStore and HbaseStore can share the code for the extrapolation logic. However, after some investigation, i found it would be hard to do (2). The main reason is that, in ObjectStore, in order to save the times to call metastore, we include lots of logic inside SQL itself, e.g., use sum and avg function in SQL to calculate the sum and the average. However, we have to write our own code in HBaseStore. And, for (1), I would suggest that we open a new JIRA to address it. Also cc'ing [~ashutoshc] to listen to his opinions. > Migrate Column Stats Extrapolation to HBaseStore > > > Key: HIVE-12960 > URL: https://issues.apache.org/jira/browse/HIVE-12960 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-12960.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13034) Add jdeb plugin to build debian
[ https://issues.apache.org/jira/browse/HIVE-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arshad Matin updated HIVE-13034: Description: It would be nice to also generate a debian as a part of build. This can be done by adding jdeb plugin to dist profile. > Add jdeb plugin to build debian > --- > > Key: HIVE-13034 > URL: https://issues.apache.org/jira/browse/HIVE-13034 > Project: Hive > Issue Type: Improvement >Reporter: Arshad Matin >Assignee: Arshad Matin > > It would be nice to also generate a debian as a part of build. This can be > done by adding jdeb plugin to dist profile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
