[jira] [Assigned] (HIVE-11899) No ADMIN PRIVILEGE need for creating and dropping temporary function.
[ https://issues.apache.org/jira/browse/HIVE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hison Huang reassigned HIVE-11899: -- Assignee: Hison Huang > No ADMIN PRIVILEGE need for creating and dropping temporary function. > - > > Key: HIVE-11899 > URL: https://issues.apache.org/jira/browse/HIVE-11899 > Project: Hive > Issue Type: Task > Components: SQLStandardAuthorization >Affects Versions: 0.13.1, 1.1.0 >Reporter: Hison Huang >Assignee: Hison Huang > Attachments: HIVE-11899-01.patch > > > SQLStandardAuthorization require ADMIN PRIVILEGE need for creating and > dropping temporary function as the following: > > create function example_add AS > > 'org.apache.hadoop.hive.contrib.udf.example.UDFExampleAdd'; > Error: Error while compiling statement: FAILED: HiveAccessControlException > Permission denied: Principal [name=admin, type=USER] does not have following > privileges for operation CREATEFUNCTION [[ADMIN PRIVILEGE] on Object > [type=DATABASE, name=default], [ADMIN PRIVILEGE] on Object [type=FUNCTION, > name=default.example_add]] (state=42000,code=4) > > drop function example_add; > Error: Error while compiling statement: FAILED: HiveAccessControlException > Permission denied: Principal [name=admin, type=USER] does not have following > privileges for operation DROPFUNCTION [[ADMIN PRIVILEGE] on Object > [type=DATABASE, name=default], [ADMIN PRIVILEGE] on Object [type=FUNCTION, > name=default.example_add]] (state=42000,code=4) > But we think it is unnecessary to create and drop temporary function. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11762) TestHCatLoaderEncryption failures when using Hadoop 2.7
[
https://issues.apache.org/jira/browse/HIVE-11762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900304#comment-14900304
]
Jason Dere commented on HIVE-11762:
---
Tests results look good. Does this one look ok [~spena]?
> TestHCatLoaderEncryption failures when using Hadoop 2.7
> ---
>
> Key: HIVE-11762
> URL: https://issues.apache.org/jira/browse/HIVE-11762
> Project: Hive
> Issue Type: Bug
> Components: Shims, Tests
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-11762.1.patch, HIVE-11762.2.patch,
> HIVE-11762.3.patch, HIVE-11762.4.patch
>
>
> When running TestHCatLoaderEncryption with -Dhadoop23.version=2.7.0, we get
> the following error during setup():
> {noformat}
> testReadDataFromEncryptedHiveTableByPig[5](org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption)
> Time elapsed: 3.648 sec <<< ERROR!
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hdfs.DFSClient.setKeyProvider(Lorg/apache/hadoop/crypto/key/KeyProviderCryptoExtension;)V
> at
> org.apache.hadoop.hive.shims.Hadoop23Shims.getMiniDfs(Hadoop23Shims.java:534)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.initEncryptionShim(TestHCatLoaderEncryption.java:252)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:200)
> {noformat}
> It looks like between Hadoop 2.6 and Hadoop 2.7, the argument to
> DFSClient.setKeyProvider() changed:
> {noformat}
>@VisibleForTesting
> - public void setKeyProvider(KeyProviderCryptoExtension provider) {
> -this.provider = provider;
> + public void setKeyProvider(KeyProvider provider) {
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11894) CBO: Calcite Operator To Hive Operator (Calcite Return Path): correct table column name in CTAS queries
[
https://issues.apache.org/jira/browse/HIVE-11894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900318#comment-14900318
]
Hive QA commented on HIVE-11894:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761366/HIVE-11894.02.patch
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 9452 tests executed
*Failed tests:*
{noformat}
TestCustomAuthentication - did not produce a TEST-*.xml file
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection
org.apache.hive.hcatalog.streaming.TestStreaming.testInterleavedTransactionBatchCommits
org.apache.hive.hcatalog.streaming.TestStreaming.testMultipleTransactionBatchCommits
org.apache.hive.hcatalog.streaming.TestStreaming.testTimeOutReaper
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchAbortAndCommit
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5357/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5357/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5357/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12761366 - PreCommit-HIVE-TRUNK-Build
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): correct table
> column name in CTAS queries
> ---
>
> Key: HIVE-11894
> URL: https://issues.apache.org/jira/browse/HIVE-11894
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-11894.01.patch, HIVE-11894.02.patch
>
>
> To repro, run lineage2.q with return path turned on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11899) No ADMIN PRIVILEGE need for creating and dropping temporary function.
[ https://issues.apache.org/jira/browse/HIVE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hison Huang updated HIVE-11899: --- Attachment: HIVE-11899-01.patch Solve the HIVE-11899 issue. > No ADMIN PRIVILEGE need for creating and dropping temporary function. > - > > Key: HIVE-11899 > URL: https://issues.apache.org/jira/browse/HIVE-11899 > Project: Hive > Issue Type: Task > Components: SQLStandardAuthorization >Affects Versions: 0.13.1, 1.1.0 >Reporter: Hison Huang > Attachments: HIVE-11899-01.patch > > > SQLStandardAuthorization require ADMIN PRIVILEGE need for creating and > dropping temporary function as the following: > > create function example_add AS > > 'org.apache.hadoop.hive.contrib.udf.example.UDFExampleAdd'; > Error: Error while compiling statement: FAILED: HiveAccessControlException > Permission denied: Principal [name=admin, type=USER] does not have following > privileges for operation CREATEFUNCTION [[ADMIN PRIVILEGE] on Object > [type=DATABASE, name=default], [ADMIN PRIVILEGE] on Object [type=FUNCTION, > name=default.example_add]] (state=42000,code=4) > > drop function example_add; > Error: Error while compiling statement: FAILED: HiveAccessControlException > Permission denied: Principal [name=admin, type=USER] does not have following > privileges for operation DROPFUNCTION [[ADMIN PRIVILEGE] on Object > [type=DATABASE, name=default], [ADMIN PRIVILEGE] on Object [type=FUNCTION, > name=default.example_add]] (state=42000,code=4) > But we think it is unnecessary to create and drop temporary function. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11866) Add framework to enable testing using LDAPServer using LDAP protocol
[
https://issues.apache.org/jira/browse/HIVE-11866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901894#comment-14901894
]
Hive QA commented on HIVE-11866:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761479/HIVE-11866.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9456 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5366/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5366/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5366/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12761479 - PreCommit-HIVE-TRUNK-Build
> Add framework to enable testing using LDAPServer using LDAP protocol
>
>
> Key: HIVE-11866
> URL: https://issues.apache.org/jira/browse/HIVE-11866
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.3.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-11866.patch
>
>
> Currently there is no unit test coverage for HS2's LDAP Atn provider using a
> LDAP Server on the backend. This prevents testing of the LDAPAtnProvider with
> some realistic usecases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11217) CTAS statements throws error, when the table is stored as ORC File format and select clause has NULL/VOID type column
[
https://issues.apache.org/jira/browse/HIVE-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901963#comment-14901963
]
Prasanth Jayachandran commented on HIVE-11217:
--
[~ychena] SemanticAnalyzer in itself is a huge chunk of code and we really
don't want to add more to it. In TypeCheckProcFactory.NullExprProcessor all you
have to do is the following to know if it is a CTAS query
{code}
boolean isCTAS =
SessionState.get().getCommandType().equals(HiveOperation.CREATETABLE_AS_SELECT.getOperationName());
{code}
> CTAS statements throws error, when the table is stored as ORC File format and
> select clause has NULL/VOID type column
> --
>
> Key: HIVE-11217
> URL: https://issues.apache.org/jira/browse/HIVE-11217
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 0.13.1
>Reporter: Gaurav Kohli
>Assignee: Yongzhi Chen
>Priority: Minor
> Attachments: HIVE-11217.1.patch, HIVE-11217.2.patch,
> HIVE-11217.3.patch, HIVE-11217.4.patch, HIVE-11271.5.patch
>
>
> If you try to use create-table-as-select (CTAS) statement and create a ORC
> File format based table, then you can't use NULL as a column value in select
> clause
> CREATE TABLE empty (x int);
> CREATE TABLE orc_table_with_null
> STORED AS ORC
> AS
> SELECT
> x,
> null
> FROM empty;
> Error:
> {quote}
> 347084 [main] ERROR hive.ql.exec.DDLTask -
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Unknown primitive type VOID
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:643)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4242)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:285)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1554)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1321)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1139)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:952)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
> at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
> at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
> at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
> at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
> at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
> at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IllegalArgumentException: Unknown primitive type VOID
> at
> org.apache.hadoop.hive.ql.io.orc.OrcStruct.createObjectInspector(OrcStruct.java:530)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcStructInspector.(OrcStruct.java:195)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcStruct.createObjectInspector(OrcStruct.java:534)
> at
>
[jira] [Updated] (HIVE-11724) WebHcat get jobs to order jobs on time order with latest at top
[ https://issues.apache.org/jira/browse/HIVE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kiran Kumar Kolli updated HIVE-11724: - Attachment: (was: HIVE-11724.5.patch) > WebHcat get jobs to order jobs on time order with latest at top > --- > > Key: HIVE-11724 > URL: https://issues.apache.org/jira/browse/HIVE-11724 > Project: Hive > Issue Type: Improvement > Components: WebHCat >Affects Versions: 0.14.0 >Reporter: Kiran Kumar Kolli >Assignee: Kiran Kumar Kolli > Attachments: HIVE-11724.1.patch, HIVE-11724.2.patch, > HIVE-11724.3.patch, HIVE-11724.4.patch > > > HIVE-5519 added pagination feature support to WebHcat. This implementation > returns the jobs lexicographically resulting in older jobs showing at the > top. > Improvement is to order them on time with latest at top. Typically latest > jobs (or running) ones are more relevant to the user. Time based ordering > with pagination makes more sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11897) JDO rollback can throw pointless exceptions
[ https://issues.apache.org/jira/browse/HIVE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901668#comment-14901668 ] Sergey Shelukhin commented on HIVE-11897: - [~jpullokkaran] [~sushanth] [~ashutoshc] ping? > JDO rollback can throw pointless exceptions > --- > > Key: HIVE-11897 > URL: https://issues.apache.org/jira/browse/HIVE-11897 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11897.patch > > > Datanucleus does a bunch of stuff before the actual rollback, with each next > step in a finally block; that way even if the prior steps fail, the rollback > should still happen. However, an exception from some questionable > pre-rollback logic like manipulating resultset after failure can affect > DirectSQL fallback -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11791) Add unit test for HIVE-10122
[ https://issues.apache.org/jira/browse/HIVE-11791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901669#comment-14901669 ] Gopal V commented on HIVE-11791: The OR change LGTM - +1. [~ashutoshc]: any comments on the test-case output? > Add unit test for HIVE-10122 > > > Key: HIVE-11791 > URL: https://issues.apache.org/jira/browse/HIVE-11791 > Project: Hive > Issue Type: Test > Components: Metastore >Affects Versions: 1.1.0 >Reporter: Illya Yalovyy >Assignee: Illya Yalovyy >Priority: Minor > Attachments: HIVE-11791.2.patch, HIVE-11791.patch > > > Unit tests for PartitionPruner.compactExpr() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11897) JDO rollback can throw pointless exceptions
[ https://issues.apache.org/jira/browse/HIVE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11897: Attachment: HIVE-11897.01.patch Improved patch. > JDO rollback can throw pointless exceptions > --- > > Key: HIVE-11897 > URL: https://issues.apache.org/jira/browse/HIVE-11897 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11897.01.patch, HIVE-11897.patch > > > Datanucleus does a bunch of stuff before the actual rollback, with each next > step in a finally block; that way even if the prior steps fail, the rollback > should still happen. However, an exception from some questionable > pre-rollback logic like manipulating resultset after failure can affect > DirectSQL fallback -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-11912) Make snappy compression default for parquet tables
[ https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho reassigned HIVE-11912: Assignee: Szehon Ho > Make snappy compression default for parquet tables > -- > > Key: HIVE-11912 > URL: https://issues.apache.org/jira/browse/HIVE-11912 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-11912.patch > > > Snappy is a popular compression codec for Parquet, and is the default in many > Parquet applications, increasing the performance. > This change would make it the default for new Hive Parquet tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11912) Make snappy compression default for parquet tables
[ https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901727#comment-14901727 ] Ferdinand Xu commented on HIVE-11912: - Hi [~szehon], why not using table properties to configure the compression type? HIVE-7858 FYI > Make snappy compression default for parquet tables > -- > > Key: HIVE-11912 > URL: https://issues.apache.org/jira/browse/HIVE-11912 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-11912.patch > > > Snappy is a popular compression codec for Parquet, and is the default in many > Parquet applications, increasing the performance. > This change would make it the default for new Hive Parquet tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11912) Make snappy compression default for parquet tables
[ https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901765#comment-14901765 ] Szehon Ho commented on HIVE-11912: -- Hey [~Ferd] I guess table properties can still be used to configure compression type, this change would not change that. I actually think something like parquet.compression should be in SerdeProperties, as its a special property unique to the serde. For example avro's serdeProperties include the location of the schema.url, which is unique to Avro. I've actually been confused for awhile why you can configure serde properties as table properties (so they seem interchange-able). But in any case, the MetaStoreUtils.getSchema appends all the Serde properties to the Table properties, such that its passed in to the code you pointed out (ParquetHiveSerde.initialize()) > Make snappy compression default for parquet tables > -- > > Key: HIVE-11912 > URL: https://issues.apache.org/jira/browse/HIVE-11912 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-11912.patch > > > Snappy is a popular compression codec for Parquet, and is the default in many > Parquet applications, increasing the performance. > This change would make it the default for new Hive Parquet tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11915) BoneCP returns closed connections from the pool
[ https://issues.apache.org/jira/browse/HIVE-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11915: Attachment: HIVE-11915.patch The patch. The optional WIP patch contains strong black magic that we can try if the weak one fails. > BoneCP returns closed connections from the pool > --- > > Key: HIVE-11915 > URL: https://issues.apache.org/jira/browse/HIVE-11915 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11915.WIP.patch, HIVE-11915.patch > > > It's a very old bug in BoneCP and it will never be fixed... There are > multiple workarounds on the internet but according to responses they are all > unreliable. We should upgrade to HikariCP (which in turn is only supported by > DN 4), meanwhile try some shamanic rituals. In this JIRA we will try a > relatively weak drum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate
[
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901809#comment-14901809
]
N Campbell commented on HIVE-9534:
--
IBM Netezza 7.2, IBM Big Insights, SAP Hana SP10, ORACLE 12 etc all return the
expected result.
> incorrect result set for query that projects a windowed aggregate
> -
>
> Key: HIVE-9534
> URL: https://issues.apache.org/jira/browse/HIVE-9534
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Reporter: N Campbell
>Assignee: Chaoyu Tang
>
> Result set returned by Hive has one row instead of 5
> {code}
> select avg(distinct tsint.csint) over () from tsint
> create table if not exists TSINT (RNUM int , CSINT smallint)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
> STORED AS TEXTFILE;
> 0|\N
> 1|-1
> 2|0
> 3|1
> 4|10
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11791) Add unit test for HIVE-10122
[ https://issues.apache.org/jira/browse/HIVE-11791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901855#comment-14901855 ] Illya Yalovyy commented on HIVE-11791: -- Please specify what is required to accept this patch. Thanks. > Add unit test for HIVE-10122 > > > Key: HIVE-11791 > URL: https://issues.apache.org/jira/browse/HIVE-11791 > Project: Hive > Issue Type: Test > Components: Metastore >Affects Versions: 1.1.0 >Reporter: Illya Yalovyy >Assignee: Illya Yalovyy >Priority: Minor > Attachments: HIVE-11791.2.patch, HIVE-11791.patch > > > Unit tests for PartitionPruner.compactExpr() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11913) Verify existence of tests for new changes in HiveQA
[ https://issues.apache.org/jira/browse/HIVE-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901909#comment-14901909 ] Brock Noland commented on HIVE-11913: - I think it just greps the patch for a few patterns. For example if a file is added in a patch which starts with Test or in our case a q file. > Verify existence of tests for new changes in HiveQA > --- > > Key: HIVE-11913 > URL: https://issues.apache.org/jira/browse/HIVE-11913 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Szehon Ho > > Would be great if HiveQA could report whether there are test files (Test*, > *Test, or qfiles) that are added, or changed. > Note not every change would need this, but it should be the best of ability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11912) Make snappy compression default for parquet tables
[
https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901908#comment-14901908
]
Brock Noland commented on HIVE-11912:
-
Good thought Ferd.
I also note that patch introduces some new imports which are not used in the
patch in {{StorageFormat}}.
> Make snappy compression default for parquet tables
> --
>
> Key: HIVE-11912
> URL: https://issues.apache.org/jira/browse/HIVE-11912
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: HIVE-11912.patch
>
>
> Snappy is a popular compression codec for Parquet, and is the default in many
> Parquet applications, increasing the performance.
> This change would make it the default for new Hive Parquet tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10785) Support aggregate push down through joins
[
https://issues.apache.org/jira/browse/HIVE-10785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-10785:
Attachment: HIVE-10785.3.patch
> Support aggregate push down through joins
> -
>
> Key: HIVE-10785
> URL: https://issues.apache.org/jira/browse/HIVE-10785
> Project: Hive
> Issue Type: Bug
> Components: CBO, Logical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-10785.2.patch, HIVE-10785.3.patch, HIVE-10785.patch
>
>
> Enable {{AggregateJoinTransposeRule}} in CBO that pushes Aggregate through
> Join operators. The rule has been extended in Calcite 1.4 to cover complex
> cases e.g. Aggregate operators comprising UDAF. The decision on whether to
> push the Aggregate through Join or not should be cost-driven.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11649) Hive UPDATE,INSERT,DELETE issue
[ https://issues.apache.org/jira/browse/HIVE-11649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900477#comment-14900477 ] Veerendra Nath Jasthi commented on HIVE-11649: -- Hi Alan, Hope everything is good at your end. Did you check my hive-site.xml ? Did I make any mistake in configurations ? Regards, Jasthi. > Hive UPDATE,INSERT,DELETE issue > --- > > Key: HIVE-11649 > URL: https://issues.apache.org/jira/browse/HIVE-11649 > Project: Hive > Issue Type: Bug > Environment: Hadoop-2.2.0 , hive-1.2.0 ,operating system > ubuntu14.04lts (64-bit) & Java 1.7 >Reporter: Veerendra Nath Jasthi >Assignee: Hive QA > Attachments: afterChange.png, beforeChange.png, hive-site.xml, > hive.log > > > have been trying to implement the UPDATE,INSERT,DELETE operations in hive > table as per link: > https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions- > > but whenever I was trying to include the properties which will do our work > i.e. > Configuration Values to Set for INSERT, UPDATE, DELETE > hive.support.concurrency true (default is false) > hive.enforce.bucketingtrue (default is false) > hive.exec.dynamic.partition.mode nonstrict (default is strict) > after that if I run show tables command on hive shell its taking 65.15 > seconds which normally runs at 0.18 seconds without the above properties. > Apart from show tables rest of the commands not giving any output i.e. its > keep on running until and unless kill the process. > Could you tell me reason for this? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11684) Implement limit pushdown through outer join in CBO
[
https://issues.apache.org/jira/browse/HIVE-11684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900559#comment-14900559
]
Hive QA commented on HIVE-11684:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761393/HIVE-11684.08.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9454 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5358/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5358/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5358/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12761393 - PreCommit-HIVE-TRUNK-Build
> Implement limit pushdown through outer join in CBO
> --
>
> Key: HIVE-11684
> URL: https://issues.apache.org/jira/browse/HIVE-11684
> Project: Hive
> Issue Type: New Feature
> Components: CBO
>Affects Versions: 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-11684.01.patch, HIVE-11684.02.patch,
> HIVE-11684.03.patch, HIVE-11684.04.patch, HIVE-11684.05.patch,
> HIVE-11684.07.patch, HIVE-11684.08.patch, HIVE-11684.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-11909) LLAP: merge master into branch
[ https://issues.apache.org/jira/browse/HIVE-11909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin resolved HIVE-11909. - Resolution: Fixed > LLAP: merge master into branch > -- > > Key: HIVE-11909 > URL: https://issues.apache.org/jira/browse/HIVE-11909 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: llap > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11915) BoneCP returns closed connections from the pool
[ https://issues.apache.org/jira/browse/HIVE-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11915: Attachment: HIVE-11915.WIP.patch > BoneCP returns closed connections from the pool > --- > > Key: HIVE-11915 > URL: https://issues.apache.org/jira/browse/HIVE-11915 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11915.WIP.patch > > > It's a very old bug in BoneCP and it will never be fixed... There are > multiple workarounds on the internet but according to responses they are all > unreliable. We should upgrade to HikariCP (which in turn is only supported by > DN 4), meanwhile try some shamanic rituals. In this JIRA we will try a > relatively weak drum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11875) JDBC Driver does not honor delegation token mechanism when readings params from ZooKeeper
[ https://issues.apache.org/jira/browse/HIVE-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901640#comment-14901640 ] Vaibhav Gumashta commented on HIVE-11875: - Just verified that delegation token mechanism works now. Ran a test oozie job, which uses delegation token while connecting. Will commit this shortly. > JDBC Driver does not honor delegation token mechanism when readings params > from ZooKeeper > - > > Key: HIVE-11875 > URL: https://issues.apache.org/jira/browse/HIVE-11875 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 1.3.0, 2.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-11875.1.patch, HIVE-11875.2.patch > > > Regression introduced in HIVE-11581. When the driver is reading connection > params from ZK, in a secure cluster if overrides the delegation token > mechanism (specified on client side) with a TGT requiring mechanism (kinit). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11898) support default partition in metasotredirectsql
[ https://issues.apache.org/jira/browse/HIVE-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901681#comment-14901681 ] Sergey Shelukhin commented on HIVE-11898: - That is interesting. > support default partition in metasotredirectsql > --- > > Key: HIVE-11898 > URL: https://issues.apache.org/jira/browse/HIVE-11898 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11898.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11897) JDO rollback can throw pointless exceptions
[ https://issues.apache.org/jira/browse/HIVE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901716#comment-14901716 ] Ashutosh Chauhan commented on HIVE-11897: - This ones better then previous, but I wonder try-with-resource might be better here : https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html ? > JDO rollback can throw pointless exceptions > --- > > Key: HIVE-11897 > URL: https://issues.apache.org/jira/browse/HIVE-11897 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11897.01.patch, HIVE-11897.patch > > > Datanucleus does a bunch of stuff before the actual rollback, with each next > step in a finally block; that way even if the prior steps fail, the rollback > should still happen. However, an exception from some questionable > pre-rollback logic like manipulating resultset after failure can affect > DirectSQL fallback -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11913) Verify existence of tests for new changes in HiveQA
[ https://issues.apache.org/jira/browse/HIVE-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-11913: - Description: Would be great if HiveQA could report whether there are test files (Test*, *Test, or qfiles) that are added, or changed. Note not every change would need this, but it should be the best of ability. > Verify existence of tests for new changes in HiveQA > --- > > Key: HIVE-11913 > URL: https://issues.apache.org/jira/browse/HIVE-11913 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Szehon Ho > > Would be great if HiveQA could report whether there are test files (Test*, > *Test, or qfiles) that are added, or changed. > Note not every change would need this, but it should be the best of ability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11875) JDBC Driver does not honor delegation token mechanism when readings params from ZooKeeper
[ https://issues.apache.org/jira/browse/HIVE-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901581#comment-14901581 ] Jason Dere commented on HIVE-11875: --- +1 > JDBC Driver does not honor delegation token mechanism when readings params > from ZooKeeper > - > > Key: HIVE-11875 > URL: https://issues.apache.org/jira/browse/HIVE-11875 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 1.3.0, 2.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-11875.1.patch, HIVE-11875.2.patch > > > Regression introduced in HIVE-11581. When the driver is reading connection > params from ZK, in a secure cluster if overrides the delegation token > mechanism (specified on client side) with a TGT requiring mechanism (kinit). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11723) Incorrect string literal escaping
[
https://issues.apache.org/jira/browse/HIVE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901728#comment-14901728
]
Yongzhi Chen commented on HIVE-11723:
-
[~szehon], I think the changes only affect values clause: something like
values('...')
This fix makes string literal in values clause more consistent with string
literals in other part of the SQL such as:
select 'a\'b' from src
The string literal above is processed in TypeCheckProcFactory as following
{noformat}
public static class StrExprProcessor implements NodeProcessor {
@Override
public Object process(Node nd, Stack stack, NodeProcessorCtx procCtx,
Object... nodeOutputs) throws SemanticException {
TypeCheckCtx ctx = (TypeCheckCtx) procCtx;
if (ctx.getError() != null) {
return null;
}
ExprNodeDesc desc = TypeCheckProcFactory.processGByExpr(nd, procCtx);
if (desc != null) {
return desc;
}
ASTNode expr = (ASTNode) nd;
String str = null;
switch (expr.getToken().getType()) {
case HiveParser.StringLiteral:
str = BaseSemanticAnalyzer.unescapeSQLString(expr.getText());
break;
{noformat}
So the change in the patch should not have backward compatible issues.
> Incorrect string literal escaping
> -
>
> Key: HIVE-11723
> URL: https://issues.apache.org/jira/browse/HIVE-11723
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.1.1, 2.0.0
>Reporter: Uri Laserson
>Assignee: Yongzhi Chen
> Attachments: HIVE-11723.1.patch
>
>
> When I execute the following queries
> {code}
> CREATE TABLE t_hive (f1 STRING);
> INSERT INTO t_hive VALUES ('Cooper\'s');
> SELECT * FROM t_hive;
> {code}
> via the Hive shell or through HiveServer2 directly (via impyla), I would
> expect that the result to be
> {code}
> Cooper's
> {code}
> but instead I actually get
> {code}
> Cooper\'s
> {code}
> Actually, I'm not sure how that {{INSERT}} query is not even a syntax error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11512) Hive LDAP Authenticator should also support full DN in Authenticate()
[ https://issues.apache.org/jira/browse/HIVE-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901762#comment-14901762 ] Lefty Leverenz commented on HIVE-11512: --- Thanks Naveen. > Hive LDAP Authenticator should also support full DN in Authenticate() > -- > > Key: HIVE-11512 > URL: https://issues.apache.org/jira/browse/HIVE-11512 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 1.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-11512.1.patch, HIVE-11512.patch > > > In certain LDAP implementation, LDAP Binding can occur using the full DN for > the user. Currently, LDAPAuthentication Provider assumes that the username > passed into Authenticate() is a short username & not a full DN. While the > initial bind works fine either way, the filter code is reliant on it being a > shortname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11913) Verify existence of tests for new changes in HiveQA
[ https://issues.apache.org/jira/browse/HIVE-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901771#comment-14901771 ] Szehon Ho commented on HIVE-11913: -- [~brocknoland] i wonder if you've ever looked into how HadoopQA does this ? > Verify existence of tests for new changes in HiveQA > --- > > Key: HIVE-11913 > URL: https://issues.apache.org/jira/browse/HIVE-11913 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Szehon Ho > > Would be great if HiveQA could report whether there are test files (Test*, > *Test, or qfiles) that are added, or changed. > Note not every change would need this, but it should be the best of ability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11891) Add basic performance logging to metastore calls
[ https://issues.apache.org/jira/browse/HIVE-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901770#comment-14901770 ] Lefty Leverenz commented on HIVE-11891: --- Thanks Brock. > Add basic performance logging to metastore calls > > > Key: HIVE-11891 > URL: https://issues.apache.org/jira/browse/HIVE-11891 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 1.0.0, 1.2.0, 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Fix For: 2.0.0 > > Attachments: HIVE-11891.patch, HIVE-11891.patch, HIVE-11891.patch, > HIVE-11891.patch > > > At present it's extremely difficult to debug slow calls to the metastore. > Ideally there would be some basic means of doing so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11910) TestHCatLoaderEncryption should shutdown created MiniDFS instance
[ https://issues.apache.org/jira/browse/HIVE-11910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-11910: -- Attachment: HIVE-11910.1.patch Attaching patch to shutdown DFS instance during tearDown() > TestHCatLoaderEncryption should shutdown created MiniDFS instance > - > > Key: HIVE-11910 > URL: https://issues.apache.org/jira/browse/HIVE-11910 > Project: Hive > Issue Type: Bug > Components: Tests >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-11910.1.patch > > > setup() creates a MiniDFS instance, but this is never shut down. On my Linux > VM this causes this test to fail with OOM/cannot create new thread/other > errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11912) Make snappy compression default for parquet tables
[ https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901705#comment-14901705 ] Szehon Ho commented on HIVE-11912: -- [~spena], [~brocknoland] would you guys be able to take a look? Thanks. > Make snappy compression default for parquet tables > -- > > Key: HIVE-11912 > URL: https://issues.apache.org/jira/browse/HIVE-11912 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-11912.patch > > > Snappy is a popular compression codec for Parquet, and is the default in many > Parquet applications, increasing the performance. > This change would make it the default for new Hive Parquet tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10814) hive on tez skew table plan wrong
[
https://issues.apache.org/jira/browse/HIVE-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yin Huai updated HIVE-10814:
Assignee: (was: Yin Huai)
> hive on tez skew table plan wrong
> ---
>
> Key: HIVE-10814
> URL: https://issues.apache.org/jira/browse/HIVE-10814
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.1.0
> Environment: hive 1.1.0 + tez 0.53
>Reporter: tangjunjie
> Attachments: skewtablequeryplan.TXT
>
>
> set hive.execution.engine=mr;
> set hive.mapred.supports.subdirectories=true;
> set hive.optimize.skewjoin.compiletime = true;
> ALTER TABLE tandem.fct_traffic_navpage_path_detl SKEWED BY
> (ordr_code,cart_prod_id) ON (('','NULL'));
> Vertex failed, vertexName=initialmap,
> vertexId=vertex_1419300485749_1514787_1_00, diagnostics=[Task failed,
> taskId=task_1419300485749_1514787_1_00_000245, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> {"parnt_ordr_id":3715999535959,"parnt_ordr_code":"3715999535959","end_user_id":163846959,"comb_prod_id":7873715,"sale_amt":99.0,"actl_sale_amt":99.0,"sale_num":1,"updt_time":"2015-05-11
> 03:58:13","etl_batch_id":0,"ds":"2015-05-10"}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.tez.mapreduce.processor.map.MapProcessor.runOldMapper(MapProcessor.java:183)
>
> at
> org.apache.tez.mapreduce.processor.map.MapProcessor.run(MapProcessor.java:126)
>
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
>
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
>
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
>
> I think this is the error,cart_prod_id col not exist in table
> univ_parnt_tranx_comb_detl
> TableScan
> alias: o
> Statistics: Num rows: 109845709 Data size: 14499651703 Basic stats: PARTIAL
> Column stats: NONE
> Filter Operator
> predicate: (not ((ordr_code = '') and (cart_prod_id = null))) (type: boolean)
> Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
> Reduce Output Operator
> key expressions: parnt_ordr_code (type: string), comb_prod_id (type: bigint)
> sort order: ++
> Map-reduce partition columns: parnt_ordr_code (type: string), comb_prod_id
> (type: bigint)
> Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
> value expressions: end_user_id (type: bigint), actl_sale_amt (type: double),
> sale_num (type: bigint), ds (type: string)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11898) support default partition in metastoredirectsql
[ https://issues.apache.org/jira/browse/HIVE-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11898: Summary: support default partition in metastoredirectsql (was: support default partition in metasotredirectsql) > support default partition in metastoredirectsql > --- > > Key: HIVE-11898 > URL: https://issues.apache.org/jira/browse/HIVE-11898 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11898.01.patch, HIVE-11898.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11915) BoneCP returns closed connections from the pool
[ https://issues.apache.org/jira/browse/HIVE-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901792#comment-14901792 ] Sergey Shelukhin commented on HIVE-11915: - [~hsubramaniyan] [~sushanth] can you take a look at the smaller patch? > BoneCP returns closed connections from the pool > --- > > Key: HIVE-11915 > URL: https://issues.apache.org/jira/browse/HIVE-11915 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11915.WIP.patch, HIVE-11915.patch > > > It's a very old bug in BoneCP and it will never be fixed... There are > multiple workarounds on the internet but according to responses they are all > unreliable. We should upgrade to HikariCP (which in turn is only supported by > DN 4), meanwhile try some shamanic rituals. In this JIRA we will try a > relatively weak drum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11911) The stats table limits are too large for innodb
[ https://issues.apache.org/jira/browse/HIVE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11911: Attachment: HIVE-11911.patch > The stats table limits are too large for innodb > --- > > Key: HIVE-11911 > URL: https://issues.apache.org/jira/browse/HIVE-11911 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > The limits were increased to a reasonable value some time ago, apparently > these values are too large for innodb due to some index limit nonsense. We > need to decrease them a little bit. > There's no need to decrease them in an upgrade script; if they were already > created successfully it's ok to have them as is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11911) The stats table limits are too large for innodb
[ https://issues.apache.org/jira/browse/HIVE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11911: Attachment: (was: HIVE-11911.patch) > The stats table limits are too large for innodb > --- > > Key: HIVE-11911 > URL: https://issues.apache.org/jira/browse/HIVE-11911 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > The limits were increased to a reasonable value some time ago, apparently > these values are too large for innodb due to some index limit nonsense. We > need to decrease them a little bit. > There's no need to decrease them in an upgrade script; if they were already > created successfully it's ok to have them as is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11912) Make snappy compression default for parquet tables
[
https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-11912:
-
Attachment: HIVE-11912.patch
Attaching a patch. Unfortunately, there's no Serde extension today to specify
default table properties, like StorageHandler.
The most logical place seemed to be StorageFormat abstraction, which is the the
rough equivalent of StorageHandler. By putting it there instead of in
AbstractSerde, we don't have to waste time initializing the Serde.
Also, there's some issues on MacOS. To workaround the Mac / Snappy issue for
Snappy version < 1.0.5 as is the case for Hadoop 2.6, HADOOP_OPTS should be set
like
{noformat}
export HADOOP_OPTS="-Dorg.xerial.snappy.tempdir=/tmp
-Dorg.xerial.snappy.lib.name=libsnappyjava.jnilib $HADOOP_OPTS"
{noformat}
> Make snappy compression default for parquet tables
> --
>
> Key: HIVE-11912
> URL: https://issues.apache.org/jira/browse/HIVE-11912
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Szehon Ho
> Attachments: HIVE-11912.patch
>
>
> Snappy is a popular compression codec for Parquet, and is the default in many
> Parquet applications, increasing the performance.
> This change would make it the default for new Hive Parquet tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11912) Make snappy compression default for parquet tables
[
https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901778#comment-14901778
]
Ferdinand Xu commented on HIVE-11912:
-
Thank you for the clarification.
The original purpose for HIVE-7858 is that ORC supports this way to configure
the table compression properties.
The patch LGTM. One minor question about the patch:
{noformat}
108 Map defaultSerdeProps =
descriptor.getDefaultSerdeProps();
109 if (defaultSerdeProps != null) {
110 serdeProps.putAll(defaultSerdeProps);
111 }
{noformat}
Is that possible the original serde properties overrided by the default value?
> Make snappy compression default for parquet tables
> --
>
> Key: HIVE-11912
> URL: https://issues.apache.org/jira/browse/HIVE-11912
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: HIVE-11912.patch
>
>
> Snappy is a popular compression codec for Parquet, and is the default in many
> Parquet applications, increasing the performance.
> This change would make it the default for new Hive Parquet tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11826) 'hadoop.proxyuser.hive.groups' configuration doesn't prevent unauthorized user to access metastore
[ https://issues.apache.org/jira/browse/HIVE-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901804#comment-14901804 ] Lefty Leverenz commented on HIVE-11826: --- Does this bug fix need any documentation? > 'hadoop.proxyuser.hive.groups' configuration doesn't prevent unauthorized > user to access metastore > -- > > Key: HIVE-11826 > URL: https://issues.apache.org/jira/browse/HIVE-11826 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-11826.2.patch, HIVE-11826.patch > > > With 'hadoop.proxyuser.hive.groups' configured in core-site.xml to certain > groups, currently if you run the job with a user not belonging to those > groups, it won't fail to access metastore. With old version hive 0.13, > actually it fails properly. > Seems HadoopThriftAuthBridge20S.java correctly call ProxyUsers.authorize() > while HadoopThriftAuthBridge23 doesn't. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11794) GBY vectorization appears to process COMPLETE reduce-side GBY incorrectly
[
https://issues.apache.org/jira/browse/HIVE-11794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901605#comment-14901605
]
Matt McCline commented on HIVE-11794:
-
+1
> GBY vectorization appears to process COMPLETE reduce-side GBY incorrectly
> -
>
> Key: HIVE-11794
> URL: https://issues.apache.org/jira/browse/HIVE-11794
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-11794.01.patch, HIVE-11794.patch
>
>
> The code in Vectorizer is as such:
> {noformat}
> boolean isMergePartial = (desc.getMode() != GroupByDesc.Mode.HASH);
> {noformat}
> then, if it's reduce side:
> {noformat}
> if (isMergePartial) {
> // Reduce Merge-Partial GROUP BY.
> // A merge-partial GROUP BY is fed by grouping by keys from
> reduce-shuffle. It is the
> // first (or root) operator for its reduce task.
>
> } else {
> // Reduce Hash GROUP BY or global aggregation.
> ...
> {noformat}
> In fact, this logic is missing the COMPLETE mode. Both from the comment:
> {noformat}
> COMPLETE: complete 1-phase aggregation: iterate, terminate
> ...
> HASH: For non-distinct the same as PARTIAL1 but use hash-table-based
> aggregation
> ...
> PARTIAL1: partial aggregation - first phase: iterate, terminatePartial
> {noformat}
> and from the explain plan like this (the query has multiple stages of
> aggregations over a union; the mapper does a partial hash aggregation for
> each side of the union, which is then followed by mergepartial, and 2nd stage
> as complete):
> {noformat}
> Map Operator Tree:
> ...
> Group By Operator
> keys: _col0 (type: int), _col1 (type: int), _col2 (type: int),
> _col3 (type: int), _col4 (type: int), _col5 (type: bigint), _col6 (type:
> bigint), _col7 (type: bigint), _col8 (type: bigint), _col9 (type: bigint),
> _col10 (type: bigint), _col11 (type: bigint), _col12 (type: bigint)
> mode: hash
> outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6,
> _col7, _col8, _col9, _col10, _col11, _col12
> Reduce Output Operator
> ...
> feeding into
> Reduce Operator Tree:
> Group By Operator
> keys: KEY._col0 (type: int), KEY._col1 (type: int), KEY._col2 (type:
> int), KEY._col3 (type: int), KEY._col4 (type: int), KEY._col5 (type: bigint),
> KEY._col6 (type: bigint), KEY._col7 (type: bigint), KEY._col8 (type: bigint),
> KEY._col9 (type: bigint), KEY._col10 (type: bigint), KEY._col11 (type:
> bigint), KEY._col12 (type: bigint)
> mode: mergepartial
> outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6,
> _col7, _col8, _col9, _col10, _col11, _col12
> Group By Operator
> aggregations: sum(_col5), sum(_col6), sum(_col7), sum(_col8),
> sum(_col9), sum(_col10), sum(_col11), sum(_col12)
> keys: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3
> (type: int), _col4 (type: int)
> mode: complete
> outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6,
> _col7, _col8, _col9, _col10, _col11, _col12
> {noformat}
> it seems like COMPLETE is actually the global aggregation, and HASH isn't (or
> may not be).
> So, it seems like reduce-side COMPLETE should be handled on the else-path of
> the above if. For map-side, it doesn't check mode at all as far as I can see.
> Not sure if additional code changes are necessary after that, it may just
> work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-11898) support default partition in metasotredirectsql
[ https://issues.apache.org/jira/browse/HIVE-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901681#comment-14901681 ] Sergey Shelukhin edited comment on HIVE-11898 at 9/22/15 12:39 AM: --- That is interesting. Looks like ORM doesn't return the default partition in this case. That may be an existing bug. was (Author: sershe): That is interesting. > support default partition in metasotredirectsql > --- > > Key: HIVE-11898 > URL: https://issues.apache.org/jira/browse/HIVE-11898 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11898.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-11898) support default partition in metasotredirectsql
[ https://issues.apache.org/jira/browse/HIVE-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901681#comment-14901681 ] Sergey Shelukhin edited comment on HIVE-11898 at 9/22/15 12:40 AM: --- That is interesting. Looks like ORM doesn't return the default partition in this case. That may be an existing bug or this fix is wrong, instead we should just ignore the thing was (Author: sershe): That is interesting. Looks like ORM doesn't return the default partition in this case. That may be an existing bug. > support default partition in metasotredirectsql > --- > > Key: HIVE-11898 > URL: https://issues.apache.org/jira/browse/HIVE-11898 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11898.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11912) Make snappy compression default for parquet tables
[ https://issues.apache.org/jira/browse/HIVE-11912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-11912: - Issue Type: Improvement (was: Bug) > Make snappy compression default for parquet tables > -- > > Key: HIVE-11912 > URL: https://issues.apache.org/jira/browse/HIVE-11912 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Szehon Ho > > Snappy is a popular compression codec for Parquet, and is the default in many > Parquet applications, increasing the performance. > This change would make it the default for new Hive Parquet tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate
[
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901818#comment-14901818
]
N Campbell commented on HIVE-9534:
--
should say Netezza 7.2 and ORACLE 12.
> incorrect result set for query that projects a windowed aggregate
> -
>
> Key: HIVE-9534
> URL: https://issues.apache.org/jira/browse/HIVE-9534
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Reporter: N Campbell
>Assignee: Chaoyu Tang
>
> Result set returned by Hive has one row instead of 5
> {code}
> select avg(distinct tsint.csint) over () from tsint
> create table if not exists TSINT (RNUM int , CSINT smallint)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
> STORED AS TEXTFILE;
> 0|\N
> 1|-1
> 2|0
> 3|1
> 4|10
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11642) LLAP: make sure tests pass #3
[ https://issues.apache.org/jira/browse/HIVE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11642: Attachment: HIVE-11642.07.patch > LLAP: make sure tests pass #3 > - > > Key: HIVE-11642 > URL: https://issues.apache.org/jira/browse/HIVE-11642 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11642.01.patch, HIVE-11642.02.patch, > HIVE-11642.03.patch, HIVE-11642.04.patch, HIVE-11642.05.patch, > HIVE-11642.06.patch, HIVE-11642.07.patch, HIVE-11642.patch > > > Tests should pass against the most recent branch and Tez 0.8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11829) Create test for HIVE-11216
[
https://issues.apache.org/jira/browse/HIVE-11829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901584#comment-14901584
]
Hive QA commented on HIVE-11829:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12756064/HIVE-11829.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9454 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5363/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5363/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5363/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12756064 - PreCommit-HIVE-TRUNK-Build
> Create test for HIVE-11216
> --
>
> Key: HIVE-11829
> URL: https://issues.apache.org/jira/browse/HIVE-11829
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 1.2.1
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-11829.1.patch
>
>
> We need tests for HIVE-11216.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11911) The stats table limits are too large for innodb
[ https://issues.apache.org/jira/browse/HIVE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11911: Attachment: HIVE-11911.patch > The stats table limits are too large for innodb > --- > > Key: HIVE-11911 > URL: https://issues.apache.org/jira/browse/HIVE-11911 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11911.patch > > > The limits were increased to a reasonable value some time ago, apparently > these values are too large for innodb due to some index limit nonsense. We > need to decrease them a little bit. > There's no need to decrease them in an upgrade script; if they were already > created successfully it's ok to have them as is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11911) The stats table limits are too large for innodb
[ https://issues.apache.org/jira/browse/HIVE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901667#comment-14901667 ] Sergey Shelukhin commented on HIVE-11911: - [~prasanth_j] can you take a look? > The stats table limits are too large for innodb > --- > > Key: HIVE-11911 > URL: https://issues.apache.org/jira/browse/HIVE-11911 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11911.patch > > > The limits were increased to a reasonable value some time ago, apparently > these values are too large for innodb due to some index limit nonsense. We > need to decrease them a little bit. > There's no need to decrease them in an upgrade script; if they were already > created successfully it's ok to have them as is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11806) Create test for HIVE-11174
[
https://issues.apache.org/jira/browse/HIVE-11806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901684#comment-14901684
]
Hive QA commented on HIVE-11806:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12755508/HIVE-11806.1.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9454 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5364/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5364/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5364/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12755508 - PreCommit-HIVE-TRUNK-Build
> Create test for HIVE-11174
> --
>
> Key: HIVE-11806
> URL: https://issues.apache.org/jira/browse/HIVE-11806
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 1.2.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Minor
> Attachments: HIVE-11806.1.patch
>
>
> We are lacking tests for HIVE-11174. Adding one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11723) Incorrect string literal escaping
[
https://issues.apache.org/jira/browse/HIVE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901688#comment-14901688
]
Szehon Ho commented on HIVE-11723:
--
[~ychena] is it a backward incompatible change? I understand the fix but am a
bit worried if it is, if it affect queries for string literals , for example
select ... where col = literal?
> Incorrect string literal escaping
> -
>
> Key: HIVE-11723
> URL: https://issues.apache.org/jira/browse/HIVE-11723
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.1.1, 2.0.0
>Reporter: Uri Laserson
>Assignee: Yongzhi Chen
> Attachments: HIVE-11723.1.patch
>
>
> When I execute the following queries
> {code}
> CREATE TABLE t_hive (f1 STRING);
> INSERT INTO t_hive VALUES ('Cooper\'s');
> SELECT * FROM t_hive;
> {code}
> via the Hive shell or through HiveServer2 directly (via impyla), I would
> expect that the result to be
> {code}
> Cooper's
> {code}
> but instead I actually get
> {code}
> Cooper\'s
> {code}
> Actually, I'm not sure how that {{INSERT}} query is not even a syntax error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11897) JDO rollback can throw pointless exceptions
[ https://issues.apache.org/jira/browse/HIVE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901698#comment-14901698 ] Ashutosh Chauhan commented on HIVE-11897: - Patch looks alright... but not sure.. if it indeed will help because catch added in patch will throw before rollback in DN can occur, so you end up in same situation. > JDO rollback can throw pointless exceptions > --- > > Key: HIVE-11897 > URL: https://issues.apache.org/jira/browse/HIVE-11897 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11897.patch > > > Datanucleus does a bunch of stuff before the actual rollback, with each next > step in a finally block; that way even if the prior steps fail, the rollback > should still happen. However, an exception from some questionable > pre-rollback logic like manipulating resultset after failure can affect > DirectSQL fallback -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-10814) hive on tez skew table plan wrong
[
https://issues.apache.org/jira/browse/HIVE-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tangjunjie reassigned HIVE-10814:
-
Assignee: Yin Huai
> hive on tez skew table plan wrong
> ---
>
> Key: HIVE-10814
> URL: https://issues.apache.org/jira/browse/HIVE-10814
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.1.0
> Environment: hive 1.1.0 + tez 0.53
>Reporter: tangjunjie
>Assignee: Yin Huai
> Attachments: skewtablequeryplan.TXT
>
>
> set hive.execution.engine=mr;
> set hive.mapred.supports.subdirectories=true;
> set hive.optimize.skewjoin.compiletime = true;
> ALTER TABLE tandem.fct_traffic_navpage_path_detl SKEWED BY
> (ordr_code,cart_prod_id) ON (('','NULL'));
> Vertex failed, vertexName=initialmap,
> vertexId=vertex_1419300485749_1514787_1_00, diagnostics=[Task failed,
> taskId=task_1419300485749_1514787_1_00_000245, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> {"parnt_ordr_id":3715999535959,"parnt_ordr_code":"3715999535959","end_user_id":163846959,"comb_prod_id":7873715,"sale_amt":99.0,"actl_sale_amt":99.0,"sale_num":1,"updt_time":"2015-05-11
> 03:58:13","etl_batch_id":0,"ds":"2015-05-10"}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.tez.mapreduce.processor.map.MapProcessor.runOldMapper(MapProcessor.java:183)
>
> at
> org.apache.tez.mapreduce.processor.map.MapProcessor.run(MapProcessor.java:126)
>
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
>
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
>
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
>
> I think this is the error,cart_prod_id col not exist in table
> univ_parnt_tranx_comb_detl
> TableScan
> alias: o
> Statistics: Num rows: 109845709 Data size: 14499651703 Basic stats: PARTIAL
> Column stats: NONE
> Filter Operator
> predicate: (not ((ordr_code = '') and (cart_prod_id = null))) (type: boolean)
> Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
> Reduce Output Operator
> key expressions: parnt_ordr_code (type: string), comb_prod_id (type: bigint)
> sort order: ++
> Map-reduce partition columns: parnt_ordr_code (type: string), comb_prod_id
> (type: bigint)
> Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
> value expressions: end_user_id (type: bigint), actl_sale_amt (type: double),
> sale_num (type: bigint), ds (type: string)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11898) support default partition in metasotredirectsql
[ https://issues.apache.org/jira/browse/HIVE-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11898: Attachment: HIVE-11898.01.patch Updated the patch to keep behavior consistent with JDO > support default partition in metasotredirectsql > --- > > Key: HIVE-11898 > URL: https://issues.apache.org/jira/browse/HIVE-11898 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11898.01.patch, HIVE-11898.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11711) Merge hbase-metastore branch to trunk
[
https://issues.apache.org/jira/browse/HIVE-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901807#comment-14901807
]
Hive QA commented on HIVE-11711:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761467/HIVE-11711.8.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9571 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5365/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5365/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5365/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12761467 - PreCommit-HIVE-TRUNK-Build
> Merge hbase-metastore branch to trunk
> -
>
> Key: HIVE-11711
> URL: https://issues.apache.org/jira/browse/HIVE-11711
> Project: Hive
> Issue Type: Sub-task
> Components: HBase Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 2.0.0
>
> Attachments: HIVE-11711.1.patch, HIVE-11711.2.patch,
> HIVE-11711.3.patch, HIVE-11711.4.patch, HIVE-11711.5.patch,
> HIVE-11711.6.patch, HIVE-11711.7.patch, HIVE-11711.8.patch
>
>
> Major development of hbase-metastore is done and it's time to merge the
> branch back into master.
> Currently hbase-metastore is only invoked when running TestMiniTezCliDriver.
> The instruction for setting up hbase-metastore is captured in
> https://cwiki.apache.org/confluence/display/Hive/HBaseMetastoreDevelopmentGuide.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11907) CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix TestJdbcDriver2
[ https://issues.apache.org/jira/browse/HIVE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-11907: --- Summary: CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix TestJdbcDriver2 (was: CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix testjdbc2) > CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix > TestJdbcDriver2 > - > > Key: HIVE-11907 > URL: https://issues.apache.org/jira/browse/HIVE-11907 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11907) CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix TestJdbcDriver2
[ https://issues.apache.org/jira/browse/HIVE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-11907: --- Description: TestJdbcDriver2 is currently failing on return path. > CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix > TestJdbcDriver2 > - > > Key: HIVE-11907 > URL: https://issues.apache.org/jira/browse/HIVE-11907 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > > TestJdbcDriver2 is currently failing on return path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11891) Add basic performance logging to metastore calls
[ https://issues.apache.org/jira/browse/HIVE-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901489#comment-14901489 ] Brock Noland commented on HIVE-11891: - This allowed us to change the dependency chain so PerfLogger could be moved to common and re-used by components which don't depend on exec. Other than the fact the code was moved, this should not impact that other bug you mention. > Add basic performance logging to metastore calls > > > Key: HIVE-11891 > URL: https://issues.apache.org/jira/browse/HIVE-11891 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 1.0.0, 1.2.0, 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Fix For: 2.0.0 > > Attachments: HIVE-11891.patch, HIVE-11891.patch, HIVE-11891.patch, > HIVE-11891.patch > > > At present it's extremely difficult to debug slow calls to the metastore. > Ideally there would be some basic means of doing so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11907) CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix TestJdbcDriver2
[ https://issues.apache.org/jira/browse/HIVE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-11907: --- Attachment: HIVE-11907.01.patch > CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix > TestJdbcDriver2 > - > > Key: HIVE-11907 > URL: https://issues.apache.org/jira/browse/HIVE-11907 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-11907.01.patch > > > TestJdbcDriver2 is currently failing on return path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11891) Add basic performance logging to metastore calls
[ https://issues.apache.org/jira/browse/HIVE-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901491#comment-14901491 ] Brock Noland commented on HIVE-11891: - I don't think we doc PerfLogger so no. It's more of a developer tool. > Add basic performance logging to metastore calls > > > Key: HIVE-11891 > URL: https://issues.apache.org/jira/browse/HIVE-11891 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 1.0.0, 1.2.0, 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Fix For: 2.0.0 > > Attachments: HIVE-11891.patch, HIVE-11891.patch, HIVE-11891.patch, > HIVE-11891.patch > > > At present it's extremely difficult to debug slow calls to the metastore. > Ideally there would be some basic means of doing so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11907) CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix TestJdbcDriver2
[ https://issues.apache.org/jira/browse/HIVE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901493#comment-14901493 ] Pengcheng Xiong commented on HIVE-11907: Note that, current testResultSetMetaData is failing on CBO old return path. > CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix > TestJdbcDriver2 > - > > Key: HIVE-11907 > URL: https://issues.apache.org/jira/browse/HIVE-11907 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-11907.01.patch > > > TestJdbcDriver2 is currently failing on return path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11110) Reorder applyPreJoinOrderingTransforms, add NotNULL/FilterMerge rules, improve Filter selectivity estimation
[
https://issues.apache.org/jira/browse/HIVE-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-0:
--
Attachment: HIVE-0.15.patch
> Reorder applyPreJoinOrderingTransforms, add NotNULL/FilterMerge rules,
> improve Filter selectivity estimation
>
>
> Key: HIVE-0
> URL: https://issues.apache.org/jira/browse/HIVE-0
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-0-10.patch, HIVE-0-11.patch,
> HIVE-0-12.patch, HIVE-0-branch-1.2.patch, HIVE-0.1.patch,
> HIVE-0.13.patch, HIVE-0.14.patch, HIVE-0.15.patch,
> HIVE-0.2.patch, HIVE-0.4.patch, HIVE-0.5.patch,
> HIVE-0.6.patch, HIVE-0.7.patch, HIVE-0.8.patch,
> HIVE-0.9.patch, HIVE-0.91.patch, HIVE-0.92.patch, HIVE-0.patch
>
>
> Query
> {code}
> select count(*)
> from store_sales
> ,store_returns
> ,date_dim d1
> ,date_dim d2
> where d1.d_quarter_name = '2000Q1'
>and d1.d_date_sk = ss_sold_date_sk
>and ss_customer_sk = sr_customer_sk
>and ss_item_sk = sr_item_sk
>and ss_ticket_number = sr_ticket_number
>and sr_returned_date_sk = d2.d_date_sk
>and d2.d_quarter_name in ('2000Q1','2000Q2','2000Q3’);
> {code}
> The store_sales table is partitioned on ss_sold_date_sk, which is also used
> in a join clause. The join clause should add a filter “filterExpr:
> ss_sold_date_sk is not null”, which should get pushed the MetaStore when
> fetching the stats. Currently this is not done in CBO planning, which results
> in the stats from __HIVE_DEFAULT_PARTITION__ to be fetched and considered in
> the optimization phase. In particular, this increases the NDV for the join
> columns and may result in wrong planning.
> Including HiveJoinAddNotNullRule in the optimization phase solves this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11908) LLAP: Merge branch to hive-2.0
[ https://issues.apache.org/jira/browse/HIVE-11908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-11908: --- Labels: TODOC-LLAP (was: ) > LLAP: Merge branch to hive-2.0 > -- > > Key: HIVE-11908 > URL: https://issues.apache.org/jira/browse/HIVE-11908 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Critical > Labels: TODOC-LLAP > > Merge LLAP branch to hive-2.0.0 (only). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11882) Fetch optimizer should stop source files traversal once it exceeds the hive.fetch.task.conversion.threshold
[ https://issues.apache.org/jira/browse/HIVE-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901504#comment-14901504 ] Gopal V commented on HIVE-11882: That looks like a missed case - assigned to you. > Fetch optimizer should stop source files traversal once it exceeds the > hive.fetch.task.conversion.threshold > --- > > Key: HIVE-11882 > URL: https://issues.apache.org/jira/browse/HIVE-11882 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 1.0.0 >Reporter: Illya Yalovyy >Assignee: Illya Yalovyy > > Hive 1.0's fetch optimizer tries to optimize queries of the form "select > from where limit " to a fetch task (see the > hive.fetch.task.conversion property). This optimization gets the lengths of > all the files in the specified partition and does some comparison against a > threshold value to determine whether it should use a fetch task or not (see > the hive.fetch.task.conversion.threshold property). This process of getting > the length of all files. One of the main problems in this optimization is the > fetch optimizer doesn't seem to stop once it exceeds the > hive.fetch.task.conversion.threshold. It works fine on HDFS, but could cause > a significant performance degradation on other supported file systems. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11882) Fetch optimizer should stop source files traversal once it exceeds the hive.fetch.task.conversion.threshold
[ https://issues.apache.org/jira/browse/HIVE-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-11882: --- Assignee: Illya Yalovyy > Fetch optimizer should stop source files traversal once it exceeds the > hive.fetch.task.conversion.threshold > --- > > Key: HIVE-11882 > URL: https://issues.apache.org/jira/browse/HIVE-11882 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 1.0.0 >Reporter: Illya Yalovyy >Assignee: Illya Yalovyy > > Hive 1.0's fetch optimizer tries to optimize queries of the form "select > from where limit " to a fetch task (see the > hive.fetch.task.conversion property). This optimization gets the lengths of > all the files in the specified partition and does some comparison against a > threshold value to determine whether it should use a fetch task or not (see > the hive.fetch.task.conversion.threshold property). This process of getting > the length of all files. One of the main problems in this optimization is the > fetch optimizer doesn't seem to stop once it exceeds the > hive.fetch.task.conversion.threshold. It works fine on HDFS, but could cause > a significant performance degradation on other supported file systems. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11724) WebHcat get jobs to order jobs on time order with latest at top
[ https://issues.apache.org/jira/browse/HIVE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kiran Kumar Kolli updated HIVE-11724: - Attachment: HIVE-11724.5.patch WebHcat capability to revers the jobs listing order based on setting templeton.jobs.listorder if set to 'reverse' > WebHcat get jobs to order jobs on time order with latest at top > --- > > Key: HIVE-11724 > URL: https://issues.apache.org/jira/browse/HIVE-11724 > Project: Hive > Issue Type: Improvement > Components: WebHCat >Affects Versions: 0.14.0 >Reporter: Kiran Kumar Kolli >Assignee: Kiran Kumar Kolli > Attachments: HIVE-11724.1.patch, HIVE-11724.2.patch, > HIVE-11724.3.patch, HIVE-11724.4.patch, HIVE-11724.5.patch > > > HIVE-5519 added pagination feature support to WebHcat. This implementation > returns the jobs lexicographically resulting in older jobs showing at the > top. > Improvement is to order them on time with latest at top. Typically latest > jobs (or running) ones are more relevant to the user. Time based ordering > with pagination makes more sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4243) Fix column names in FileSinkOperator
[ https://issues.apache.org/jira/browse/HIVE-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901505#comment-14901505 ] Prasanth Jayachandran commented on HIVE-4243: - Left some more comments in RB. > Fix column names in FileSinkOperator > > > Key: HIVE-4243 > URL: https://issues.apache.org/jira/browse/HIVE-4243 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 1.3.0, 2.0.0 >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-4243.patch, HIVE-4243.patch, HIVE-4243.patch, > HIVE-4243.tmp.patch > > > All of the ObjectInspectors given to SerDe's by FileSinkOperator have virtual > column names. Since the files are part of tables, Hive knows the column > names. For self-describing file formats like ORC, having the real column > names will improve the understandability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11911) The stats table limits are too large for innodb
[ https://issues.apache.org/jira/browse/HIVE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901925#comment-14901925 ] Prasanth Jayachandran commented on HIVE-11911: -- JDBCStats publishing is having all sorts of issues with key prefix. I don't think thats a problem with counter and fs based stats. Do we really need to support it? [~ashutoshc] Any thoughts on this? Excerpt from this link http://dba.stackexchange.com/questions/76469/mysql-varchar-length-and-performance "It has some consequences for InnoDB, too. Index size is restricted to 3072 bytes and single column indexes, to 767 bytes." > The stats table limits are too large for innodb > --- > > Key: HIVE-11911 > URL: https://issues.apache.org/jira/browse/HIVE-11911 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11911.patch > > > The limits were increased to a reasonable value some time ago, apparently > these values are too large for innodb due to some index limit nonsense. We > need to decrease them a little bit. > There's no need to decrease them in an upgrade script; if they were already > created successfully it's ok to have them as is. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11724) WebHcat get jobs to order jobs on time order with latest at top
[
https://issues.apache.org/jira/browse/HIVE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901986#comment-14901986
]
Hive QA commented on HIVE-11724:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761491/HIVE-11724.4.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9453 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5367/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5367/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5367/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12761491 - PreCommit-HIVE-TRUNK-Build
> WebHcat get jobs to order jobs on time order with latest at top
> ---
>
> Key: HIVE-11724
> URL: https://issues.apache.org/jira/browse/HIVE-11724
> Project: Hive
> Issue Type: Improvement
> Components: WebHCat
>Affects Versions: 0.14.0
>Reporter: Kiran Kumar Kolli
>Assignee: Kiran Kumar Kolli
> Attachments: HIVE-11724.1.patch, HIVE-11724.2.patch,
> HIVE-11724.3.patch, HIVE-11724.4.patch
>
>
> HIVE-5519 added pagination feature support to WebHcat. This implementation
> returns the jobs lexicographically resulting in older jobs showing at the
> top.
> Improvement is to order them on time with latest at top. Typically latest
> jobs (or running) ones are more relevant to the user. Time based ordering
> with pagination makes more sense.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11785) Support escaping carriage return and new line for LazySimpleSerDe
[
https://issues.apache.org/jira/browse/HIVE-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11785:
Summary: Support escaping carriage return and new line for LazySimpleSerDe
(was: Support escaping carriage return and new line)
> Support escaping carriage return and new line for LazySimpleSerDe
> -
>
> Key: HIVE-11785
> URL: https://issues.apache.org/jira/browse/HIVE-11785
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: test.parquet
>
>
> Create the table and perform the queries as follows. You will see different
> results when the setting changes. Seems both present incorrect results.
> The expected result should be:
> {noformat}
> 1 newline
> here
> 2 carriage return
> 3 both
> here
> {noformat}
> {noformat}
> hive> create table repo (lvalue int, charstring string) stored as parquet;
> OK
> Time taken: 0.34 seconds
> hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo;
> Loading data to table default.repo
> chgrp: changing ownership of
> 'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not
> belong to hive
> Table default.repo stats: [numFiles=1, numRows=0, totalSize=610,
> rawDataSize=0]
> OK
> Time taken: 0.732 seconds
> hive> set hive.fetch.task.conversion=more;
> hive> select * from repo;
> OK
> 1 newline
> here
> here carriage return
> 3 both
> here
> Time taken: 0.253 seconds, Fetched: 3 row(s)
> hive> set hive.fetch.task.conversion=none;
> hive> select * from repo;
> Query ID = root_20150909113535_e081db8b-ccd9-4c44-aad9-d990ffb8edf3
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1441752031022_0006, Tracking URL =
> http://host-10-17-81-63.coe.cloudera.com:8088/proxy/application_1441752031022_0006/
> Kill Command =
> /opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/lib/hadoop/bin/hadoop job
> -kill job_1441752031022_0006
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers: > 0
> 2015-09-09 11:35:54,127 Stage-1 map = 0%, reduce = 0%
> 2015-09-09 11:36:04,664 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.98
> sec
> MapReduce Total cumulative CPU time: 2 seconds 980 msec
> Ended Job = job_1441752031022_0006
> MapReduce Jobs Launched:
> Stage-Stage-1: Map: 1 Cumulative CPU: 2.98 sec HDFS Read: 4251 HDFS
> Write: 51 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 980 msec
> OK
> 1 newline
> NULL NULL
> 2 carriage return
> NULL NULL
> 3 both
> NULL NULL
> Time taken: 25.131 seconds, Fetched: 6 row(s)
> hive>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11785) Support escaping carriage return and new line for LazySimpleSerDe
[
https://issues.apache.org/jira/browse/HIVE-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11785:
Issue Type: Improvement (was: Bug)
> Support escaping carriage return and new line for LazySimpleSerDe
> -
>
> Key: HIVE-11785
> URL: https://issues.apache.org/jira/browse/HIVE-11785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11785.patch, test.parquet
>
>
> Create the table and perform the queries as follows. You will see different
> results when the setting changes.
> The expected result should be:
> {noformat}
> 1 newline
> here
> 2 carriage return
> 3 both
> here
> {noformat}
> {noformat}
> hive> create table repo (lvalue int, charstring string) stored as parquet;
> OK
> Time taken: 0.34 seconds
> hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo;
> Loading data to table default.repo
> chgrp: changing ownership of
> 'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not
> belong to hive
> Table default.repo stats: [numFiles=1, numRows=0, totalSize=610,
> rawDataSize=0]
> OK
> Time taken: 0.732 seconds
> hive> set hive.fetch.task.conversion=more;
> hive> select * from repo;
> OK
> 1 newline
> here
> here carriage return
> 3 both
> here
> Time taken: 0.253 seconds, Fetched: 3 row(s)
> hive> set hive.fetch.task.conversion=none;
> hive> select * from repo;
> Query ID = root_20150909113535_e081db8b-ccd9-4c44-aad9-d990ffb8edf3
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1441752031022_0006, Tracking URL =
> http://host-10-17-81-63.coe.cloudera.com:8088/proxy/application_1441752031022_0006/
> Kill Command =
> /opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/lib/hadoop/bin/hadoop job
> -kill job_1441752031022_0006
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers: > 0
> 2015-09-09 11:35:54,127 Stage-1 map = 0%, reduce = 0%
> 2015-09-09 11:36:04,664 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.98
> sec
> MapReduce Total cumulative CPU time: 2 seconds 980 msec
> Ended Job = job_1441752031022_0006
> MapReduce Jobs Launched:
> Stage-Stage-1: Map: 1 Cumulative CPU: 2.98 sec HDFS Read: 4251 HDFS
> Write: 51 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 980 msec
> OK
> 1 newline
> NULL NULL
> 2 carriage return
> NULL NULL
> 3 both
> NULL NULL
> Time taken: 25.131 seconds, Fetched: 6 row(s)
> hive>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11724) WebHcat get jobs to order jobs on time order with latest at top
[ https://issues.apache.org/jira/browse/HIVE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kiran Kumar Kolli updated HIVE-11724: - Attachment: HIVE-11724.3.patch Templeton capability to list the jobs in reverse order if setting templeton.jobs.listorder set to 'reverse' > WebHcat get jobs to order jobs on time order with latest at top > --- > > Key: HIVE-11724 > URL: https://issues.apache.org/jira/browse/HIVE-11724 > Project: Hive > Issue Type: Improvement > Components: WebHCat >Affects Versions: 0.14.0 >Reporter: Kiran Kumar Kolli >Assignee: Kiran Kumar Kolli > Attachments: HIVE-11724.1.patch, HIVE-11724.2.patch, > HIVE-11724.3.patch > > > HIVE-5519 added pagination feature support to WebHcat. This implementation > returns the jobs lexicographically resulting in older jobs showing at the > top. > Improvement is to order them on time with latest at top. Typically latest > jobs (or running) ones are more relevant to the user. Time based ordering > with pagination makes more sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11785) Support escaping carriage return and new line
[
https://issues.apache.org/jira/browse/HIVE-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11785:
Summary: Support escaping carriage return and new line (was: Carriage
return and new line are processed differently when hive.fetch.task.conversion
is set to none)
> Support escaping carriage return and new line
> -
>
> Key: HIVE-11785
> URL: https://issues.apache.org/jira/browse/HIVE-11785
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: test.parquet
>
>
> Create the table and perform the queries as follows. You will see different
> results when the setting changes. Seems both present incorrect results.
> The expected result should be:
> {noformat}
> 1 newline
> here
> 2 carriage return
> 3 both
> here
> {noformat}
> {noformat}
> hive> create table repo (lvalue int, charstring string) stored as parquet;
> OK
> Time taken: 0.34 seconds
> hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo;
> Loading data to table default.repo
> chgrp: changing ownership of
> 'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not
> belong to hive
> Table default.repo stats: [numFiles=1, numRows=0, totalSize=610,
> rawDataSize=0]
> OK
> Time taken: 0.732 seconds
> hive> set hive.fetch.task.conversion=more;
> hive> select * from repo;
> OK
> 1 newline
> here
> here carriage return
> 3 both
> here
> Time taken: 0.253 seconds, Fetched: 3 row(s)
> hive> set hive.fetch.task.conversion=none;
> hive> select * from repo;
> Query ID = root_20150909113535_e081db8b-ccd9-4c44-aad9-d990ffb8edf3
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1441752031022_0006, Tracking URL =
> http://host-10-17-81-63.coe.cloudera.com:8088/proxy/application_1441752031022_0006/
> Kill Command =
> /opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/lib/hadoop/bin/hadoop job
> -kill job_1441752031022_0006
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers: > 0
> 2015-09-09 11:35:54,127 Stage-1 map = 0%, reduce = 0%
> 2015-09-09 11:36:04,664 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.98
> sec
> MapReduce Total cumulative CPU time: 2 seconds 980 msec
> Ended Job = job_1441752031022_0006
> MapReduce Jobs Launched:
> Stage-Stage-1: Map: 1 Cumulative CPU: 2.98 sec HDFS Read: 4251 HDFS
> Write: 51 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 980 msec
> OK
> 1 newline
> NULL NULL
> 2 carriage return
> NULL NULL
> 3 both
> NULL NULL
> Time taken: 25.131 seconds, Fetched: 6 row(s)
> hive>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11785) Support escaping carriage return and new line for LazySimpleSerDe
[
https://issues.apache.org/jira/browse/HIVE-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11785:
Description:
Create the table and perform the queries as follows. You will see different
results when the setting changes.
The expected result should be:
{noformat}
1 newline
here
2 carriage return
3 both
here
{noformat}
{noformat}
hive> create table repo (lvalue int, charstring string) stored as parquet;
OK
Time taken: 0.34 seconds
hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo;
Loading data to table default.repo
chgrp: changing ownership of
'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not
belong to hive
Table default.repo stats: [numFiles=1, numRows=0, totalSize=610, rawDataSize=0]
OK
Time taken: 0.732 seconds
hive> set hive.fetch.task.conversion=more;
hive> select * from repo;
OK
1 newline
here
herecarriage return
3 both
here
Time taken: 0.253 seconds, Fetched: 3 row(s)
hive> set hive.fetch.task.conversion=none;
hive> select * from repo;
Query ID = root_20150909113535_e081db8b-ccd9-4c44-aad9-d990ffb8edf3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1441752031022_0006, Tracking URL =
http://host-10-17-81-63.coe.cloudera.com:8088/proxy/application_1441752031022_0006/
Kill Command =
/opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/lib/hadoop/bin/hadoop job
-kill job_1441752031022_0006
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2015-09-09 11:35:54,127 Stage-1 map = 0%, reduce = 0%
2015-09-09 11:36:04,664 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.98
sec
MapReduce Total cumulative CPU time: 2 seconds 980 msec
Ended Job = job_1441752031022_0006
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 2.98 sec HDFS Read: 4251 HDFS Write:
51 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 980 msec
OK
1 newline
NULLNULL
2 carriage return
NULLNULL
3 both
NULLNULL
Time taken: 25.131 seconds, Fetched: 6 row(s)
hive>
{noformat}
was:
Create the table and perform the queries as follows. You will see different
results when the setting changes. Seems both present incorrect results.
The expected result should be:
{noformat}
1 newline
here
2 carriage return
3 both
here
{noformat}
{noformat}
hive> create table repo (lvalue int, charstring string) stored as parquet;
OK
Time taken: 0.34 seconds
hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo;
Loading data to table default.repo
chgrp: changing ownership of
'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not
belong to hive
Table default.repo stats: [numFiles=1, numRows=0, totalSize=610, rawDataSize=0]
OK
Time taken: 0.732 seconds
hive> set hive.fetch.task.conversion=more;
hive> select * from repo;
OK
1 newline
here
herecarriage return
3 both
here
Time taken: 0.253 seconds, Fetched: 3 row(s)
hive> set hive.fetch.task.conversion=none;
hive> select * from repo;
Query ID = root_20150909113535_e081db8b-ccd9-4c44-aad9-d990ffb8edf3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1441752031022_0006, Tracking URL =
http://host-10-17-81-63.coe.cloudera.com:8088/proxy/application_1441752031022_0006/
Kill Command =
/opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/lib/hadoop/bin/hadoop job
-kill job_1441752031022_0006
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2015-09-09 11:35:54,127 Stage-1 map = 0%, reduce = 0%
2015-09-09 11:36:04,664 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.98
sec
MapReduce Total cumulative CPU time: 2 seconds 980 msec
Ended Job = job_1441752031022_0006
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 2.98 sec HDFS Read: 4251 HDFS Write:
51 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 980 msec
OK
1 newline
NULLNULL
2 carriage return
NULLNULL
3 both
NULLNULL
Time taken: 25.131 seconds, Fetched: 6 row(s)
hive>
{noformat}
> Support escaping carriage return and new line for LazySimpleSerDe
> -
>
> Key: HIVE-11785
> URL: https://issues.apache.org/jira/browse/HIVE-11785
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: test.parquet
>
>
> Create the table and perform the queries as follows. You will see different
> results when the setting changes.
> The expected result should be:
> {noformat}
> 1 newline
> here
> 2 carriage return
> 3 both
> here
> {noformat}
> {noformat}
> hive> create
[jira] [Resolved] (HIVE-11649) Hive UPDATE,INSERT,DELETE issue
[ https://issues.apache.org/jira/browse/HIVE-11649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved HIVE-11649. --- Resolution: Invalid > Hive UPDATE,INSERT,DELETE issue > --- > > Key: HIVE-11649 > URL: https://issues.apache.org/jira/browse/HIVE-11649 > Project: Hive > Issue Type: Bug > Environment: Hadoop-2.2.0 , hive-1.2.0 ,operating system > ubuntu14.04lts (64-bit) & Java 1.7 >Reporter: Veerendra Nath Jasthi >Assignee: Hive QA > Attachments: afterChange.png, beforeChange.png, hive-site.xml, > hive.log > > > have been trying to implement the UPDATE,INSERT,DELETE operations in hive > table as per link: > https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions- > > but whenever I was trying to include the properties which will do our work > i.e. > Configuration Values to Set for INSERT, UPDATE, DELETE > hive.support.concurrency true (default is false) > hive.enforce.bucketingtrue (default is false) > hive.exec.dynamic.partition.mode nonstrict (default is strict) > after that if I run show tables command on hive shell its taking 65.15 > seconds which normally runs at 0.18 seconds without the above properties. > Apart from show tables rest of the commands not giving any output i.e. its > keep on running until and unless kill the process. > Could you tell me reason for this? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11649) Hive UPDATE,INSERT,DELETE issue
[ https://issues.apache.org/jira/browse/HIVE-11649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900890#comment-14900890 ] Alan Gates commented on HIVE-11649: --- hive.txn.manager org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager Set to org.apache.hadoop.hive.ql.lockmgr.DbTxnManager as part of turning on Hive transactions, which also requires appropriate settings for hive.compactor.initiator.on, hive.compactor.worker.threads, hive.support.concurrency (true), hive.enforce.bucketing (true), and hive.exec.dynamic.partition.mode (nonstrict). The default DummyTxnManager replicates pre-Hive-0.13 behavior and provides no transactions. This should be DbTxnManager if you want to use update and delete. It won't work with DummyTxnManager. The reason it's taking a long time is when you set the txn manager to Dummy and the concurrency to true you get the zookeeper lock manager, which is taking time trying to connect to zookeeper. > Hive UPDATE,INSERT,DELETE issue > --- > > Key: HIVE-11649 > URL: https://issues.apache.org/jira/browse/HIVE-11649 > Project: Hive > Issue Type: Bug > Environment: Hadoop-2.2.0 , hive-1.2.0 ,operating system > ubuntu14.04lts (64-bit) & Java 1.7 >Reporter: Veerendra Nath Jasthi >Assignee: Hive QA > Attachments: afterChange.png, beforeChange.png, hive-site.xml, > hive.log > > > have been trying to implement the UPDATE,INSERT,DELETE operations in hive > table as per link: > https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions- > > but whenever I was trying to include the properties which will do our work > i.e. > Configuration Values to Set for INSERT, UPDATE, DELETE > hive.support.concurrency true (default is false) > hive.enforce.bucketingtrue (default is false) > hive.exec.dynamic.partition.mode nonstrict (default is strict) > after that if I run show tables command on hive shell its taking 65.15 > seconds which normally runs at 0.18 seconds without the above properties. > Apart from show tables rest of the commands not giving any output i.e. its > keep on running until and unless kill the process. > Could you tell me reason for this? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-11723) Incorrect string literal escaping
[
https://issues.apache.org/jira/browse/HIVE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen reassigned HIVE-11723:
---
Assignee: Yongzhi Chen
> Incorrect string literal escaping
> -
>
> Key: HIVE-11723
> URL: https://issues.apache.org/jira/browse/HIVE-11723
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Uri Laserson
>Assignee: Yongzhi Chen
>
> When I execute the following queries
> {code}
> CREATE TABLE t_hive (f1 STRING);
> INSERT INTO t_hive VALUES ('Cooper\'s');
> SELECT * FROM t_hive;
> {code}
> via the Hive shell or through HiveServer2 directly (via impyla), I would
> expect that the result to be
> {code}
> Cooper's
> {code}
> but instead I actually get
> {code}
> Cooper\'s
> {code}
> Actually, I'm not sure how that {{INSERT}} query is not even a syntax error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11785) Support escaping carriage return and new line for LazySimpleSerDe
[
https://issues.apache.org/jira/browse/HIVE-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11785:
Attachment: HIVE-11785.patch
> Support escaping carriage return and new line for LazySimpleSerDe
> -
>
> Key: HIVE-11785
> URL: https://issues.apache.org/jira/browse/HIVE-11785
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-11785.patch, test.parquet
>
>
> Create the table and perform the queries as follows. You will see different
> results when the setting changes.
> The expected result should be:
> {noformat}
> 1 newline
> here
> 2 carriage return
> 3 both
> here
> {noformat}
> {noformat}
> hive> create table repo (lvalue int, charstring string) stored as parquet;
> OK
> Time taken: 0.34 seconds
> hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo;
> Loading data to table default.repo
> chgrp: changing ownership of
> 'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not
> belong to hive
> Table default.repo stats: [numFiles=1, numRows=0, totalSize=610,
> rawDataSize=0]
> OK
> Time taken: 0.732 seconds
> hive> set hive.fetch.task.conversion=more;
> hive> select * from repo;
> OK
> 1 newline
> here
> here carriage return
> 3 both
> here
> Time taken: 0.253 seconds, Fetched: 3 row(s)
> hive> set hive.fetch.task.conversion=none;
> hive> select * from repo;
> Query ID = root_20150909113535_e081db8b-ccd9-4c44-aad9-d990ffb8edf3
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1441752031022_0006, Tracking URL =
> http://host-10-17-81-63.coe.cloudera.com:8088/proxy/application_1441752031022_0006/
> Kill Command =
> /opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/lib/hadoop/bin/hadoop job
> -kill job_1441752031022_0006
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers: > 0
> 2015-09-09 11:35:54,127 Stage-1 map = 0%, reduce = 0%
> 2015-09-09 11:36:04,664 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.98
> sec
> MapReduce Total cumulative CPU time: 2 seconds 980 msec
> Ended Job = job_1441752031022_0006
> MapReduce Jobs Launched:
> Stage-Stage-1: Map: 1 Cumulative CPU: 2.98 sec HDFS Read: 4251 HDFS
> Write: 51 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 980 msec
> OK
> 1 newline
> NULL NULL
> 2 carriage return
> NULL NULL
> 3 both
> NULL NULL
> Time taken: 25.131 seconds, Fetched: 6 row(s)
> hive>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11785) Support escaping carriage return and new line for LazySimpleSerDe
[
https://issues.apache.org/jira/browse/HIVE-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11785:
Issue Type: New Feature (was: Improvement)
> Support escaping carriage return and new line for LazySimpleSerDe
> -
>
> Key: HIVE-11785
> URL: https://issues.apache.org/jira/browse/HIVE-11785
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11785.patch, test.parquet
>
>
> Create the table and perform the queries as follows. You will see different
> results when the setting changes.
> The expected result should be:
> {noformat}
> 1 newline
> here
> 2 carriage return
> 3 both
> here
> {noformat}
> {noformat}
> hive> create table repo (lvalue int, charstring string) stored as parquet;
> OK
> Time taken: 0.34 seconds
> hive> load data inpath '/tmp/repo/test.parquet' overwrite into table repo;
> Loading data to table default.repo
> chgrp: changing ownership of
> 'hdfs://nameservice1/user/hive/warehouse/repo/test.parquet': User does not
> belong to hive
> Table default.repo stats: [numFiles=1, numRows=0, totalSize=610,
> rawDataSize=0]
> OK
> Time taken: 0.732 seconds
> hive> set hive.fetch.task.conversion=more;
> hive> select * from repo;
> OK
> 1 newline
> here
> here carriage return
> 3 both
> here
> Time taken: 0.253 seconds, Fetched: 3 row(s)
> hive> set hive.fetch.task.conversion=none;
> hive> select * from repo;
> Query ID = root_20150909113535_e081db8b-ccd9-4c44-aad9-d990ffb8edf3
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1441752031022_0006, Tracking URL =
> http://host-10-17-81-63.coe.cloudera.com:8088/proxy/application_1441752031022_0006/
> Kill Command =
> /opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/lib/hadoop/bin/hadoop job
> -kill job_1441752031022_0006
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers: > 0
> 2015-09-09 11:35:54,127 Stage-1 map = 0%, reduce = 0%
> 2015-09-09 11:36:04,664 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.98
> sec
> MapReduce Total cumulative CPU time: 2 seconds 980 msec
> Ended Job = job_1441752031022_0006
> MapReduce Jobs Launched:
> Stage-Stage-1: Map: 1 Cumulative CPU: 2.98 sec HDFS Read: 4251 HDFS
> Write: 51 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 980 msec
> OK
> 1 newline
> NULL NULL
> 2 carriage return
> NULL NULL
> 3 both
> NULL NULL
> Time taken: 25.131 seconds, Fetched: 6 row(s)
> hive>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11783) Extending HPL/SQL parser
[ https://issues.apache.org/jira/browse/HIVE-11783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900968#comment-14900968 ] Alan Gates commented on HIVE-11783: --- +1 > Extending HPL/SQL parser > > > Key: HIVE-11783 > URL: https://issues.apache.org/jira/browse/HIVE-11783 > Project: Hive > Issue Type: Improvement > Components: hpl/sql >Reporter: Dmitry Tolpeko >Assignee: Dmitry Tolpeko > Attachments: HIVE-11783.1.patch > > > Need to extend procedural SQL parser and synchronize code base by adding > PART_COUNT, PART_COUNT_BY functions as well as CMP ROW_COUNT, CMP SUM and > COPY TO HDFS statements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11723) Incorrect string literal escaping
[
https://issues.apache.org/jira/browse/HIVE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901040#comment-14901040
]
Yongzhi Chen commented on HIVE-11723:
-
String literals in the values clause should not only be processed by strip
quotes, they need to unescape escaped chars too.
Attach the patch to fix this.
> Incorrect string literal escaping
> -
>
> Key: HIVE-11723
> URL: https://issues.apache.org/jira/browse/HIVE-11723
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Uri Laserson
>Assignee: Yongzhi Chen
> Attachments: HIVE-11723.1.patch
>
>
> When I execute the following queries
> {code}
> CREATE TABLE t_hive (f1 STRING);
> INSERT INTO t_hive VALUES ('Cooper\'s');
> SELECT * FROM t_hive;
> {code}
> via the Hive shell or through HiveServer2 directly (via impyla), I would
> expect that the result to be
> {code}
> Cooper's
> {code}
> but instead I actually get
> {code}
> Cooper\'s
> {code}
> Actually, I'm not sure how that {{INSERT}} query is not even a syntax error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11723) Incorrect string literal escaping
[
https://issues.apache.org/jira/browse/HIVE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-11723:
Attachment: HIVE-11723.1.patch
> Incorrect string literal escaping
> -
>
> Key: HIVE-11723
> URL: https://issues.apache.org/jira/browse/HIVE-11723
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Uri Laserson
>Assignee: Yongzhi Chen
> Attachments: HIVE-11723.1.patch
>
>
> When I execute the following queries
> {code}
> CREATE TABLE t_hive (f1 STRING);
> INSERT INTO t_hive VALUES ('Cooper\'s');
> SELECT * FROM t_hive;
> {code}
> via the Hive shell or through HiveServer2 directly (via impyla), I would
> expect that the result to be
> {code}
> Cooper's
> {code}
> but instead I actually get
> {code}
> Cooper\'s
> {code}
> Actually, I'm not sure how that {{INSERT}} query is not even a syntax error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11748) HivePreparedStatement's setTimestamp() does not quote value as required
[
https://issues.apache.org/jira/browse/HIVE-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901110#comment-14901110
]
Sergio Peña commented on HIVE-11748:
Got it.
Thanks [~awsmithson] for the explanation.
Then there's no issue right now, but we're fixing the API to avoid an issue in
the future.
The change looks good. +1
> HivePreparedStatement's setTimestamp() does not quote value as required
> ---
>
> Key: HIVE-11748
> URL: https://issues.apache.org/jira/browse/HIVE-11748
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Reporter: Angus Smithson
>Assignee: Angus Smithson
> Attachments: HIVE-11748.2.patch, HIVE-11748.patch
>
>
> [HivePreparedStatement.setTimestamp(int parameterIndex, Timestamp
> x)|https://hive.apache.org/javadocs/r1.2.1/api/org/apache/hive/jdbc/HivePreparedStatement.html#setTimestamp(int,%20java.sql.Timestamp)]
> does not quote the Timestamp value when generating the HQL statement,
> resulting in a HiveSqlException on execution.
> h5. Reproducing
> If we add the following unit test to
> {{itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java}}:
> {code}
> @Test
> public void testPrepareSetTimestamp() throws SQLException, ParseException {
> String sql = String.format("SELECT * FROM %s WHERE c20 = ?",
> dataTypeTableName);
> try (PreparedStatement ps = con.prepareStatement(sql)) {
> Timestamp timestamp = new Timestamp(new
> SimpleDateFormat("-MM-dd").parse("2013-01-01").getTime());
> ps.setTimestamp(1, timestamp);
> try (ResultSet resultSet = ps.executeQuery()) {
> assertTrue(resultSet.next());
> assertEquals("2013-01-01", resultSet.getString(20));
> }
> }
> }
> {code}
> The test fails:
> {noformat}
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: ParseException line 1:55 cannot recognize input near '00'
> ':' '00' in expression specification
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:205)
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:401)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:310)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1150)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1136)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:109)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:180)
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:405)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:392)
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:261)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:509)
> at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hive.jdbc.HiveConnection$SynchronizedHandler.invoke(HiveConnection.java:1400)
> at com.sun.proxy.$Proxy32.ExecuteStatement(Unknown Source)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:246)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:378)
> at
> org.apache.hive.jdbc.HivePreparedStatement.executeQuery(HivePreparedStatement.java:109)
> at
> org.apache.hive.jdbc.TestJdbcDriver2.testPrepareSetTimestamp(TestJdbcDriver2.java:2395)
> {noformat}
> The failure is because the following HQL is generated/executed by calling
> toString() on the Timestamp value:
> {noformat}
> SELECT * FROM testdatatypetable WHERE c20 = 2013-01-01 00:00:00.0
> {noformat}
> We should be quoting the value of Timestamp.toString(), so that the following
> HQL is generated/executed:
> {noformat}
> SELECT * FROM testdatatypetable WHERE c20 = '2013-01-01 00:00:00.0'
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11889) Add unit test for HIVE-11449
[ https://issues.apache.org/jira/browse/HIVE-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-11889: -- Fix Version/s: 1.3.0 > Add unit test for HIVE-11449 > > > Key: HIVE-11889 > URL: https://issues.apache.org/jira/browse/HIVE-11889 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.3.0, 2.0.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-11889.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11889) Add unit test for HIVE-11449
[ https://issues.apache.org/jira/browse/HIVE-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901113#comment-14901113 ] Jason Dere commented on HIVE-11889: --- Also committed to branch-1 > Add unit test for HIVE-11449 > > > Key: HIVE-11889 > URL: https://issues.apache.org/jira/browse/HIVE-11889 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.3.0, 2.0.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-11889.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11785) Support escaping carriage return and new line for LazySimpleSerDe
[
https://issues.apache.org/jira/browse/HIVE-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901085#comment-14901085
]
Hive QA commented on HIVE-11785:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761445/HIVE-11785.patch
{color:red}ERROR:{color} -1 due to 154 failed/errored test(s), 9453 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_reordering_values
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_spark4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_outer_join_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_partlvl
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_tbllvl
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_display_colstats_tbllvl
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_join_breaktask
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_fouter_join_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters_overlap
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_louter_join_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_outer_join_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pointlookup2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pointlookup3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_union_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_vc
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_allchildsarenull
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_push_or
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_query_result_fileformat
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rand_partitionpruner1
[jira] [Commented] (HIVE-11711) Merge hbase-metastore branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901090#comment-14901090 ] Vaibhav Gumashta commented on HIVE-11711: - +1 > Merge hbase-metastore branch to trunk > - > > Key: HIVE-11711 > URL: https://issues.apache.org/jira/browse/HIVE-11711 > Project: Hive > Issue Type: Sub-task > Components: HBase Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 2.0.0 > > Attachments: HIVE-11711.1.patch, HIVE-11711.2.patch, > HIVE-11711.3.patch, HIVE-11711.4.patch, HIVE-11711.5.patch, > HIVE-11711.6.patch, HIVE-11711.7.patch > > > Major development of hbase-metastore is done and it's time to merge the > branch back into master. > Currently hbase-metastore is only invoked when running TestMiniTezCliDriver. > The instruction for setting up hbase-metastore is captured in > https://cwiki.apache.org/confluence/display/Hive/HBaseMetastoreDevelopmentGuide. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11724) WebHcat get jobs to order jobs on time order with latest at top
[
https://issues.apache.org/jira/browse/HIVE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901091#comment-14901091
]
Hive QA commented on HIVE-11724:
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761450/HIVE-11724.3.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5360/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5360/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5360/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-5360/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
c092563..e1eebff branch-1 -> origin/branch-1
92b42ae..4ff5b25 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 92b42ae HIVE-11843: Add 'sort by c' to Parquet PPD q-tests to
avoid different output issues with hadoop-1 (Sergio Pena, reviewed by Ferdinand
Xu)
+ git clean -f -d
Removing ql/src/test/queries/clientpositive/escape_crlf.q
Removing ql/src/test/results/clientpositive/escape_crlf.q.out
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
+ git reset --hard origin/master
HEAD is now at 4ff5b25 HIVE-11889: Add unit test for HIVE-11449 (Wei Zheng via
Jason Dere)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
patch: Only garbage was found in the patch input.
patch: Only garbage was found in the patch input.
patch: Only garbage was found in the patch input.
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12761450 - PreCommit-HIVE-TRUNK-Build
> WebHcat get jobs to order jobs on time order with latest at top
> ---
>
> Key: HIVE-11724
> URL: https://issues.apache.org/jira/browse/HIVE-11724
> Project: Hive
> Issue Type: Improvement
> Components: WebHCat
>Affects Versions: 0.14.0
>Reporter: Kiran Kumar Kolli
>Assignee: Kiran Kumar Kolli
> Fix For: 0.14.0
>
> Attachments: HIVE-11724.1.patch, HIVE-11724.2.patch,
> HIVE-11724.3.patch
>
>
> HIVE-5519 added pagination feature support to WebHcat. This implementation
> returns the jobs lexicographically resulting in older jobs showing at the
> top.
> Improvement is to order them on time with latest at top. Typically latest
> jobs (or running) ones are more relevant to the user. Time based ordering
> with pagination makes more sense.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11889) Add unit test for HIVE-11449
[ https://issues.apache.org/jira/browse/HIVE-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901049#comment-14901049 ] Wei Zheng commented on HIVE-11889: -- Test failure unrelated. > Add unit test for HIVE-11449 > > > Key: HIVE-11889 > URL: https://issues.apache.org/jira/browse/HIVE-11889 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.3.0, 2.0.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-11889.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901060#comment-14901060
]
Eugene Koifman commented on HIVE-11902:
---
+1 pending tests
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11711) Merge hbase-metastore branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901072#comment-14901072 ] Thejas M Nair commented on HIVE-11711: -- +1 I agree we should put metastore config parameters in the metaVars in HiveConf. However, I think we can also do that in a follow up patch if this one merges fine right now. > Merge hbase-metastore branch to trunk > - > > Key: HIVE-11711 > URL: https://issues.apache.org/jira/browse/HIVE-11711 > Project: Hive > Issue Type: Sub-task > Components: HBase Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 2.0.0 > > Attachments: HIVE-11711.1.patch, HIVE-11711.2.patch, > HIVE-11711.3.patch, HIVE-11711.4.patch, HIVE-11711.5.patch, > HIVE-11711.6.patch, HIVE-11711.7.patch > > > Major development of hbase-metastore is done and it's time to merge the > branch back into master. > Currently hbase-metastore is only invoked when running TestMiniTezCliDriver. > The instruction for setting up hbase-metastore is captured in > https://cwiki.apache.org/confluence/display/Hive/HBaseMetastoreDevelopmentGuide. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10165) Improve hive-hcatalog-streaming extensibility and support updates and deletes.
[
https://issues.apache.org/jira/browse/HIVE-10165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901096#comment-14901096
]
Eugene Koifman commented on HIVE-10165:
---
[~teabot] I've noticed that BucketIdResolverImpl.computeBucketId is different
from ReduceSinkOperator.computeBucketNumber. The seed for hash computation is
0 in one case and 1 in the other.
Also, the ABS computation is coded differently - not sure whether this can make
a difference.
> Improve hive-hcatalog-streaming extensibility and support updates and deletes.
> --
>
> Key: HIVE-10165
> URL: https://issues.apache.org/jira/browse/HIVE-10165
> Project: Hive
> Issue Type: Improvement
> Components: HCatalog
>Affects Versions: 1.2.0
>Reporter: Elliot West
>Assignee: Elliot West
> Labels: TODOC2.0, streaming_api
> Fix For: 2.0.0
>
> Attachments: HIVE-10165.0.patch, HIVE-10165.10.patch,
> HIVE-10165.4.patch, HIVE-10165.5.patch, HIVE-10165.6.patch,
> HIVE-10165.7.patch, HIVE-10165.9.patch, mutate-system-overview.png
>
>
> h3. Overview
> I'd like to extend the
> [hive-hcatalog-streaming|https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest]
> API so that it also supports the writing of record updates and deletes in
> addition to the already supported inserts.
> h3. Motivation
> We have many Hadoop processes outside of Hive that merge changed facts into
> existing datasets. Traditionally we achieve this by: reading in a
> ground-truth dataset and a modified dataset, grouping by a key, sorting by a
> sequence and then applying a function to determine inserted, updated, and
> deleted rows. However, in our current scheme we must rewrite all partitions
> that may potentially contain changes. In practice the number of mutated
> records is very small when compared with the records contained in a
> partition. This approach results in a number of operational issues:
> * Excessive amount of write activity required for small data changes.
> * Downstream applications cannot robustly read these datasets while they are
> being updated.
> * Due to scale of the updates (hundreds or partitions) the scope for
> contention is high.
> I believe we can address this problem by instead writing only the changed
> records to a Hive transactional table. This should drastically reduce the
> amount of data that we need to write and also provide a means for managing
> concurrent access to the data. Our existing merge processes can read and
> retain each record's {{ROW_ID}}/{{RecordIdentifier}} and pass this through to
> an updated form of the hive-hcatalog-streaming API which will then have the
> required data to perform an update or insert in a transactional manner.
> h3. Benefits
> * Enables the creation of large-scale dataset merge processes
> * Opens up Hive transactional functionality in an accessible manner to
> processes that operate outside of Hive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-11902:
--
Attachment: (was: HIVE-11902.patch)
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-11902:
--
Attachment: HIVE-11902.patch
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-11902:
--
Attachment: HIVE-11902.patch
Attaching a patch for review. Also minor change in the logging message. cc
[~ekoifman]
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11831) TXN tables in Oracle should be created with ROWDEPENDENCIES
[ https://issues.apache.org/jira/browse/HIVE-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900960#comment-14900960 ] Alan Gates commented on HIVE-11831: --- +1, patch itself looks good. When you say no possibility of upgrade, do we need to build a recommended procedure for people to migrate their data so they can fix the deadlocks if they are already using this feature in Oracle? Something like: # pause the world # create temp table # move contents of existing table to temp table # drop existing table # create new table with ROWDEPENDENCIES # move data back from temp table > TXN tables in Oracle should be created with ROWDEPENDENCIES > --- > > Key: HIVE-11831 > URL: https://issues.apache.org/jira/browse/HIVE-11831 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11831.01.patch, HIVE-11831.patch > > > These frequently-updated tables may otherwise suffer from spurious deadlocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11826) 'hadoop.proxyuser.hive.groups' configuration doesn't prevent unauthorized user to access metastore
[ https://issues.apache.org/jira/browse/HIVE-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901032#comment-14901032 ] Aihua Xu commented on HIVE-11826: - Thanks Chao. > 'hadoop.proxyuser.hive.groups' configuration doesn't prevent unauthorized > user to access metastore > -- > > Key: HIVE-11826 > URL: https://issues.apache.org/jira/browse/HIVE-11826 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-11826.2.patch, HIVE-11826.patch > > > With 'hadoop.proxyuser.hive.groups' configured in core-site.xml to certain > groups, currently if you run the job with a user not belonging to those > groups, it won't fail to access metastore. With old version hive 0.13, > actually it fails properly. > Seems HadoopThriftAuthBridge20S.java correctly call ProxyUsers.authorize() > while HadoopThriftAuthBridge23 doesn't. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11512) Hive LDAP Authenticator should also support full DN in Authenticate()
[ https://issues.apache.org/jira/browse/HIVE-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900740#comment-14900740 ] Naveen Gangam commented on HIVE-11512: -- [~leftylev] I dont think there is anything to be documented. > Hive LDAP Authenticator should also support full DN in Authenticate() > -- > > Key: HIVE-11512 > URL: https://issues.apache.org/jira/browse/HIVE-11512 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 1.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-11512.1.patch, HIVE-11512.patch > > > In certain LDAP implementation, LDAP Binding can occur using the full DN for > the user. Currently, LDAPAuthentication Provider assumes that the username > passed into Authenticate() is a short username & not a full DN. While the > initial bind works fine either way, the filter code is reliant on it being a > shortname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
