[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.13.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.11.patch, > HIVE-18118.12.patch, HIVE-18118.13.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, > HIVE-18118.6.patch, HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.6.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, HIVE-18118.6.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19581) view do not support unicode characters well
[
https://issues.apache.org/jira/browse/HIVE-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-19581:
--
Attachment: HIVE-19581.4.patch
> view do not support unicode characters well
> ---
>
> Key: HIVE-19581
> URL: https://issues.apache.org/jira/browse/HIVE-19581
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: kai
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-19581.1.patch, HIVE-19581.2.patch,
> HIVE-19581.3.patch, HIVE-19581.4.patch, explain.png, metastore.png
>
>
> create table t_test (name ,string) ;
> insert into table t_test VALUES ('李四');
> create view t_view_test as select * from t_test where name='李四';
> when select * from t_view_test no records return
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.10.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.2.patch, HIVE-18118.3.patch, > HIVE-18118.4.patch, HIVE-18118.5.patch, HIVE-18118.6.patch, > HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.11.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, > HIVE-18118.6.patch, HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19581) view do not support unicode characters well
[
https://issues.apache.org/jira/browse/HIVE-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-19581:
--
Attachment: HIVE-19581.5.patch
> view do not support unicode characters well
> ---
>
> Key: HIVE-19581
> URL: https://issues.apache.org/jira/browse/HIVE-19581
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: kai
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-19581.1.patch, HIVE-19581.2.patch,
> HIVE-19581.3.patch, HIVE-19581.4.patch, HIVE-19581.5.patch, explain.png,
> metastore.png
>
>
> create table t_test (name ,string) ;
> insert into table t_test VALUES ('李四');
> create view t_view_test as select * from t_test where name='李四';
> when select * from t_view_test no records return
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.12.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.11.patch, > HIVE-18118.12.patch, HIVE-18118.2.patch, HIVE-18118.3.patch, > HIVE-18118.4.patch, HIVE-18118.5.patch, HIVE-18118.6.patch, > HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19971) TestRuntimeStats.testCleanup() is flaky
[ https://issues.apache.org/jira/browse/HIVE-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-19971: -- Attachment: HIVE-19971.3.patch > TestRuntimeStats.testCleanup() is flaky > --- > > Key: HIVE-19971 > URL: https://issues.apache.org/jira/browse/HIVE-19971 > Project: Hive > Issue Type: Task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-19971.1.patch, HIVE-19971.2.patch, > HIVE-19971.3.patch > > > This test is timing dependent and sometimes fails. [You can see that it > sometimes fails in otherwise clean > runs|https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestRuntimeStats%22]. > The test inserts a stat, sleeps for 2 seconds, inserts another stat, then > deletes stats that are older than 1 second. The test asserts that exactly one > stat is deleted. If the deletion is slow for some reason (perhaps a GC?) then > 2 stats will be deleted and the test will fail. The trouble is that the 1 > second window is too small to work consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.8.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, > HIVE-18118.6.patch, HIVE-18118.7.patch, HIVE-18118.8.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20000) woooohoo20000ooooooo
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524193#comment-16524193
]
Andrew Sherman commented on HIVE-2:
---
Well done Hive team for 2 jiras!
> whoo2ooo
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: New Feature
> Components: Hive
>Affects Versions: All Versions
>Reporter: Prasanth Jayachandran
>Priority: Blocker
> Fix For: All Versions
>
>
> {code:java}
> ::: ::: ::: :::
> :+::+::+: :+::+: :+::+: :+::+: :+:
> +:+ +:+ :+:++:+ :+:++:+ :+:++:+ :+:+
> +#+ +#+ + +:++#+ + +:++#+ + +:++#+ + +:+
> +#+ +#+# +#++#+# +#++#+# +#++#+# +#+
> #+# #+# #+##+# #+##+# #+##+# #+#
> ## ### ### ### ###
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.9.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, > HIVE-18118.6.patch, HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.10.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.2.patch, HIVE-18118.3.patch, HIVE-18118.4.patch, > HIVE-18118.5.patch, HIVE-18118.6.patch, HIVE-18118.7.patch, > HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.11.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.11.patch, > HIVE-18118.2.patch, HIVE-18118.3.patch, HIVE-18118.4.patch, > HIVE-18118.5.patch, HIVE-18118.6.patch, HIVE-18118.7.patch, > HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19971) TestRuntimeStats.testCleanup() is flaky
[ https://issues.apache.org/jira/browse/HIVE-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-19971: -- Attachment: HIVE-19971.4.patch > TestRuntimeStats.testCleanup() is flaky > --- > > Key: HIVE-19971 > URL: https://issues.apache.org/jira/browse/HIVE-19971 > Project: Hive > Issue Type: Task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-19971.1.patch, HIVE-19971.2.patch, > HIVE-19971.3.patch, HIVE-19971.4.patch > > > This test is timing dependent and sometimes fails. [You can see that it > sometimes fails in otherwise clean > runs|https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestRuntimeStats%22]. > The test inserts a stat, sleeps for 2 seconds, inserts another stat, then > deletes stats that are older than 1 second. The test asserts that exactly one > stat is deleted. If the deletion is slow for some reason (perhaps a GC?) then > 2 stats will be deleted and the test will fail. The trouble is that the 1 > second window is too small to work consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19581) view do not support unicode characters well
[
https://issues.apache.org/jira/browse/HIVE-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-19581:
--
Attachment: HIVE-19581.2.patch
> view do not support unicode characters well
> ---
>
> Key: HIVE-19581
> URL: https://issues.apache.org/jira/browse/HIVE-19581
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: kai
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-19581.1.patch, HIVE-19581.2.patch, explain.png,
> metastore.png
>
>
> create table t_test (name ,string) ;
> insert into table t_test VALUES ('李四');
> create view t_view_test as select * from t_test where name='李四';
> when select * from t_view_test no records return
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19971) TestRuntimeStats.testCleanup() is flaky
[ https://issues.apache.org/jira/browse/HIVE-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-19971: -- Attachment: HIVE-19971.2.patch > TestRuntimeStats.testCleanup() is flaky > --- > > Key: HIVE-19971 > URL: https://issues.apache.org/jira/browse/HIVE-19971 > Project: Hive > Issue Type: Task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-19971.1.patch, HIVE-19971.2.patch > > > This test is timing dependent and sometimes fails. [You can see that it > sometimes fails in otherwise clean > runs|https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestRuntimeStats%22]. > The test inserts a stat, sleeps for 2 seconds, inserts another stat, then > deletes stats that are older than 1 second. The test asserts that exactly one > stat is deleted. If the deletion is slow for some reason (perhaps a GC?) then > 2 stats will be deleted and the test will fail. The trouble is that the 1 > second window is too small to work consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19971) TestRuntimeStats.testCleanup() is flaky
[ https://issues.apache.org/jira/browse/HIVE-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-19971: -- Attachment: HIVE-19971.1.patch Status: Patch Available (was: Open) > TestRuntimeStats.testCleanup() is flaky > --- > > Key: HIVE-19971 > URL: https://issues.apache.org/jira/browse/HIVE-19971 > Project: Hive > Issue Type: Task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-19971.1.patch > > > This test is timing dependent and sometimes fails. [You can see that it > sometimes fails in otherwise clean > runs|https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestRuntimeStats%22]. > The test inserts a stat, sleeps for 2 seconds, inserts another stat, then > deletes stats that are older than 1 second. The test asserts that exactly one > stat is deleted. If the deletion is slow for some reason (perhaps a GC?) then > 2 stats will be deleted and the test will fail. The trouble is that the 1 > second window is too small to work consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19391) RowContainer.setupWriter hangs when the file creation failed
[
https://issues.apache.org/jira/browse/HIVE-19391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned HIVE-19391:
-
Assignee: (was: Andrew Sherman)
> RowContainer.setupWriter hangs when the file creation failed
>
>
> Key: HIVE-19391
> URL: https://issues.apache.org/jira/browse/HIVE-19391
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.0.0
>Reporter: John Doe
>Priority: Major
>
> The while loop in the RowContainer.setupWriter function hangs endlessly when
> the file creation failed, causing the delete() returns false.
> The file creation failure can be caused by different reasons, e.g., disk full.
> Here is the code snippet.
>
> {code:java}
> protected void setupWriter() throws HiveException {
> try {
> if ( tmpFile != null ) {
> return;
> }
> String suffix = ".tmp";
> if (this.keyObject != null) {
> suffix = "." + this.keyObject.toString() + suffix;
> }
> while (true) {
> parentFile = File.createTempFile("hive-rowcontainer", ""); //file
> creation failed
> boolean success = parentFile.delete() && parentFile.mkdir(); //delete
> returns false
> if (success) {
> break;
> }
> LOG.debug("retry creating tmp row-container directory...");
> }
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19391) RowContainer.setupWriter hangs when the file creation failed
[
https://issues.apache.org/jira/browse/HIVE-19391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520820#comment-16520820
]
Andrew Sherman commented on HIVE-19391:
---
That scary while loop was removed in [HIVE-13303] so I think this bug does not
exist after hive 2.0
> RowContainer.setupWriter hangs when the file creation failed
>
>
> Key: HIVE-19391
> URL: https://issues.apache.org/jira/browse/HIVE-19391
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.0.0
>Reporter: John Doe
>Assignee: Andrew Sherman
>Priority: Major
>
> The while loop in the RowContainer.setupWriter function hangs endlessly when
> the file creation failed, causing the delete() returns false.
> The file creation failure can be caused by different reasons, e.g., disk full.
> Here is the code snippet.
>
> {code:java}
> protected void setupWriter() throws HiveException {
> try {
> if ( tmpFile != null ) {
> return;
> }
> String suffix = ".tmp";
> if (this.keyObject != null) {
> suffix = "." + this.keyObject.toString() + suffix;
> }
> while (true) {
> parentFile = File.createTempFile("hive-rowcontainer", ""); //file
> creation failed
> boolean success = parentFile.delete() && parentFile.mkdir(); //delete
> returns false
> if (success) {
> break;
> }
> LOG.debug("retry creating tmp row-container directory...");
> }
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19971) TestRuntimeStats.testCleanup() is flaky
[ https://issues.apache.org/jira/browse/HIVE-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned HIVE-19971: - > TestRuntimeStats.testCleanup() is flaky > --- > > Key: HIVE-19971 > URL: https://issues.apache.org/jira/browse/HIVE-19971 > Project: Hive > Issue Type: Task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > This test is timing dependent and sometimes fails. [You can see that it > sometimes fails in otherwise clean > runs|https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestRuntimeStats%22]. > The test inserts a stat, sleeps for 2 seconds, inserts another stat, then > deletes stats that are older than 1 second. The test asserts that exactly one > stat is deleted. If the deletion is slow for some reason (perhaps a GC?) then > 2 stats will be deleted and the test will fail. The trouble is that the 1 > second window is too small to work consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19392) unsynchronized index causes DataInputByteBuffer$Buffer.read hangs
[ https://issues.apache.org/jira/browse/HIVE-19392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned HIVE-19392: - Assignee: (was: Andrew Sherman) > unsynchronized index causes DataInputByteBuffer$Buffer.read hangs > - > > Key: HIVE-19392 > URL: https://issues.apache.org/jira/browse/HIVE-19392 > Project: Hive > Issue Type: Bug > Components: SQL >Affects Versions: 1.0.0 >Reporter: John Doe >Priority: Minor > > In DataInputByteBuffer$Buffer class, the fields bidx and buffers, etc are > unsynchronized when used in read() and reset() function. In certain > circumstances, e.g., the reset() is invoked in a loop, the unsynchronized > bidx and buffers can trigger a concurrency bug. > This bug also shows in > [Hadoop-15429|https://issues.apache.org/jira/browse/HADOOP-15429]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19392) unsynchronized index causes DataInputByteBuffer$Buffer.read hangs
[ https://issues.apache.org/jira/browse/HIVE-19392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520811#comment-16520811 ] Andrew Sherman commented on HIVE-19392: --- Hi [~dustinday] I think DataInputByteBuffer is not thread safe (and the parent class java.io.DataInputStream is explicitly not thread safe) but it does not seem to be used by multiple threads. Did you see a failure in hive/tez? > unsynchronized index causes DataInputByteBuffer$Buffer.read hangs > - > > Key: HIVE-19392 > URL: https://issues.apache.org/jira/browse/HIVE-19392 > Project: Hive > Issue Type: Bug > Components: SQL >Affects Versions: 1.0.0 >Reporter: John Doe >Assignee: Andrew Sherman >Priority: Minor > > In DataInputByteBuffer$Buffer class, the fields bidx and buffers, etc are > unsynchronized when used in read() and reset() function. In certain > circumstances, e.g., the reset() is invoked in a loop, the unsynchronized > bidx and buffers can trigger a concurrency bug. > This bug also shows in > [Hadoop-15429|https://issues.apache.org/jira/browse/HADOOP-15429]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19581) view do not support unicode characters well
[
https://issues.apache.org/jira/browse/HIVE-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-19581:
--
Attachment: HIVE-19581.3.patch
> view do not support unicode characters well
> ---
>
> Key: HIVE-19581
> URL: https://issues.apache.org/jira/browse/HIVE-19581
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: kai
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-19581.1.patch, HIVE-19581.2.patch,
> HIVE-19581.3.patch, explain.png, metastore.png
>
>
> create table t_test (name ,string) ;
> insert into table t_test VALUES ('李四');
> create view t_view_test as select * from t_test where name='李四';
> when select * from t_view_test no records return
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.7.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, > HIVE-18118.6.patch, HIVE-18118.7.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19987) Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by Spark
[ https://issues.apache.org/jira/browse/HIVE-19987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-19987: -- Issue Type: Sub-task (was: Task) Parent: HIVE-18116 > Add logging of runtime statistics indicating when Hdfs Erasure Coding is used > by Spark > -- > > Key: HIVE-19987 > URL: https://issues.apache.org/jira/browse/HIVE-19987 > Project: Hive > Issue Type: Sub-task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19986) Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by MR
[ https://issues.apache.org/jira/browse/HIVE-19986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned HIVE-19986: - > Add logging of runtime statistics indicating when Hdfs Erasure Coding is used > by MR > --- > > Key: HIVE-19986 > URL: https://issues.apache.org/jira/browse/HIVE-19986 > Project: Hive > Issue Type: Task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19986) Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by MR
[ https://issues.apache.org/jira/browse/HIVE-19986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-19986: -- Issue Type: Sub-task (was: Task) Parent: HIVE-18116 > Add logging of runtime statistics indicating when Hdfs Erasure Coding is used > by MR > --- > > Key: HIVE-19986 > URL: https://issues.apache.org/jira/browse/HIVE-19986 > Project: Hive > Issue Type: Sub-task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19987) Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by Spark
[ https://issues.apache.org/jira/browse/HIVE-19987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned HIVE-19987: - > Add logging of runtime statistics indicating when Hdfs Erasure Coding is used > by Spark > -- > > Key: HIVE-19987 > URL: https://issues.apache.org/jira/browse/HIVE-19987 > Project: Hive > Issue Type: Task >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19404) Revise DDL Task Result Logging
[
https://issues.apache.org/jira/browse/HIVE-19404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520912#comment-16520912
]
Andrew Sherman commented on HIVE-19404:
---
Proposed new logging:
2018-06-22T17:00:17,048 INFO [bd2d27a2-0a87-4c9c-827a-628afe6fb741 main]
exec.DDLTask: Found 1 database(s) matching the SHOW DATABASES statement.
2018-06-22T17:00:17,078 INFO [bd2d27a2-0a87-4c9c-827a-628afe6fb741 main]
exec.DDLTask: Found 1 function(s) matching the SHOW FUNCTIONS statement.
2018-06-22T17:00:17,124 DEBUG [bd2d27a2-0a87-4c9c-827a-628afe6fb741 main]
exec.DDLTask: Found 2 tables/view(s) matching the SHOW TABLES/VIEWS statement.
2018-06-22T17:00:17,164 DEBUG [bd2d27a2-0a87-4c9c-827a-628afe6fb741 main]
exec.DDLTask: Found 0 tables/view(s) matching the SHOW TABLES/VIEWS statement.
2018-06-22T17:00:17,212 INFO [bd2d27a2-0a87-4c9c-827a-628afe6fb741 main]
exec.DDLTask: Found 1 table(s) matching the SHOW TABLE EXTENDED statement.
> Revise DDL Task Result Logging
> --
>
> Key: HIVE-19404
> URL: https://issues.apache.org/jira/browse/HIVE-19404
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Trivial
> Labels: noob
>
> There is some logging in {{DDLTask}} that can be made better:
> {code}
> 2018-05-03 03:08:32,524 INFO hive.ql.exec.DDLTask:
> [HiveServer2-Background-Pool: Thread-101980]: results : 706
> {code}
> This logging should either be demoted to _debug_ level logging and/or
> requires additional context.
> {code}
> 2018-05-03 03:08:32,524 INFO hive.ql.exec.DDLTask:
> [HiveServer2-Background-Pool: Thread-101980]: Found 706 tables that match the
> SHOW DATABASE statement
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19966) Compile error when using open-jdk
[
https://issues.apache.org/jira/browse/HIVE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520554#comment-16520554
]
Andrew Sherman commented on HIVE-19966:
---
[~vihangk1] Are you sure it was OpenJdk8 and not 9? Other projects are having
to make changes for jdk9, e.g. HADOOP-12760
> Compile error when using open-jdk
> -
>
> Key: HIVE-19966
> URL: https://issues.apache.org/jira/browse/HIVE-19966
> Project: Hive
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> When you compile Hive using open-jdk-8 you see this error
> {noformat}
> hive/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java:[72,16]
> sun.misc.Cleaner is internal proprietary API and may be removed in a future
> release
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hive-exec
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.4.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.5.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.3.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524512#comment-16524512 ] Andrew Sherman commented on HIVE-18118: --- HIVE-18118.11.patch is the same as HIVE-18118.12.patch so the test failures form the latter are just further examples of flakiness. > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.11.patch, > HIVE-18118.12.patch, HIVE-18118.2.patch, HIVE-18118.3.patch, > HIVE-18118.4.patch, HIVE-18118.5.patch, HIVE-18118.6.patch, > HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20000) woooohoo20000ooooooo
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524234#comment-16524234
]
Andrew Sherman commented on HIVE-2:
---
Looks similar to [HIVE-1]
> whoo2ooo
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: New Feature
> Components: Hive
>Affects Versions: All Versions
>Reporter: Prasanth Jayachandran
>Assignee: Hive QA
>Priority: Blocker
> Fix For: All Versions
>
>
> {code:java}
> ::: ::: ::: :::
> :+::+::+: :+::+: :+::+: :+::+: :+:
> +:+ +:+ :+:++:+ :+:++:+ :+:++:+ :+:+
> +#+ +#+ + +:++#+ + +:++#+ + +:++#+ + +:+
> +#+ +#+# +#++#+# +#++#+# +#++#+# +#+
> #+# #+# #+##+# #+##+# #+##+# #+#
> ## ### ### ### ###
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19404) Revise DDL Task Result Logging
[
https://issues.apache.org/jira/browse/HIVE-19404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-19404:
--
Attachment: HIVE-19404.1.patch
Status: Patch Available (was: Open)
> Revise DDL Task Result Logging
> --
>
> Key: HIVE-19404
> URL: https://issues.apache.org/jira/browse/HIVE-19404
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-19404.1.patch
>
>
> There is some logging in {{DDLTask}} that can be made better:
> {code}
> 2018-05-03 03:08:32,524 INFO hive.ql.exec.DDLTask:
> [HiveServer2-Background-Pool: Thread-101980]: results : 706
> {code}
> This logging should either be demoted to _debug_ level logging and/or
> requires additional context.
> {code}
> 2018-05-03 03:08:32,524 INFO hive.ql.exec.DDLTask:
> [HiveServer2-Background-Pool: Thread-101980]: Found 706 tables that match the
> SHOW DATABASE statement
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524388#comment-16524388 ] Andrew Sherman commented on HIVE-18118: --- [~stakiar] This has a clean test run and is ready to go, can you take a look and push if you agree? Thanks > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.11.patch, > HIVE-18118.12.patch, HIVE-18118.2.patch, HIVE-18118.3.patch, > HIVE-18118.4.patch, HIVE-18118.5.patch, HIVE-18118.6.patch, > HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20055) SQL injection via metastore ACID APIs (and maybe queries, although that's unlikely)
[
https://issues.apache.org/jira/browse/HIVE-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530404#comment-16530404
]
Andrew Sherman commented on HIVE-20055:
---
I'm not sure I completely understand this jira, but often then best way to
avoid sql injection attacks is to code jdbc using prepared statements.
> SQL injection via metastore ACID APIs (and maybe queries, although that's
> unlikely)
> ---
>
> Key: HIVE-20055
> URL: https://issues.apache.org/jira/browse/HIVE-20055
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Thejas M Nair
>Priority: Major
>
> [~thejas] asked me to create this JIRA based on my earlier email :)
> {noformat}
> This might be doable with a specially crafted query, I’m not sure what APIs
> calls have what checks (e.g. via Hive parser) that would prevent the below.
> However, for remote metastore (default on many clusters currently, afaik it’s
> the default for ACID) we expose thrift API that accepts strings e.g.
> get_valid_write_ids.
> That passes the string table names to TxnHandler::getValidWriteIdsForTable,
> that inserts them into the query string w/quoteString call; quoteString
> doesn’t do any validation.
> Some ready made delete statements also exist e.g. "delete from REPL_TXN_MAP
> where RTM_SRC_TXN_ID = " + sourceTxnId + " and RTM_REPL_POLICY = " +
> quoteString(rqst.getReplPolicy());
> I think my replication policy might be {' OR '1' = '1} ;)
> So, SQL injection might be possible thru these APIs.
> I wonder if this class should be switched to parameter based execution?
> DirectSQL could be used as an example, although that uses DataNucleus direct
> sql feature… at least we need some checks on these.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20030) Fix Java compile errors that show up in IntelliJ from ConvertJoinMapJoin.java and AnnotateRunTimeStatsOptimizer.java
[
https://issues.apache.org/jira/browse/HIVE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530252#comment-16530252
]
Andrew Sherman commented on HIVE-20030:
---
[~kgyrtkirk] that is interesting, thanks for pointing out HIVE-20008.
With your change the problem is now worse :-) When I rebase, clean build, and
then try to run with IntelliJ I see
{code}
/Users/asherman/git/asf/hive/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Warning:(239, 45) java: org.apache.hadoop.hive.ql.udf.generic.GenericUDFHash in
org.apache.hadoop.hive.ql.udf.generic has been deprecated
Error:(525, 70) java: incompatible types:
java.util.List> cannot be converted to
java.util.List>
Warning:(1819, 48) java: HIVEMAPREDMODE in
org.apache.hadoop.hive.conf.HiveConf.ConfVars has been deprecated
Warning:(10924, 72) java: org.apache.hadoop.hive.ql.udf.generic.GenericUDFHash
in org.apache.hadoop.hive.ql.udf.generic has been deprecated
Error:(12207, 52) java: incompatible types:
java.util.List> cannot be converted to
java.util.List>
Error:(12307, 30) java: incompatible types:
java.util.List> cannot be converted to
java.util.List>
Warning:(13260, 43) java: isDir() in org.apache.hadoop.fs.FileStatus has been
deprecated
Warning:(13526, 61) java: HIVE_GROUPBY_ORDERBY_POSITION_ALIAS in
org.apache.hadoop.hive.conf.HiveConf.ConfVars has been deprecated
Error:(14766, 49) java: incompatible types:
java.util.List> cannot be converted to
java.util.List>
{code}
In the code there are a lot of places where we use so
removing those is a much larger task than converting the few Operator to
Operator>.
> Fix Java compile errors that show up in IntelliJ from ConvertJoinMapJoin.java
> and AnnotateRunTimeStatsOptimizer.java
>
>
> Key: HIVE-20030
> URL: https://issues.apache.org/jira/browse/HIVE-20030
> Project: Hive
> Issue Type: Task
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-20030.1.patch
>
>
> For some reason the Java compiler in IntellJ is more strict that the Oracle
> jdk compiler. Maybe this is something that can be configured away, but as it
> is simple I propose to make the code more type correct.
> {code}
> /Users/asherman/git/asf/hive2/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java
> Error:(613, 24) java: no suitable method found for
> findOperatorsUpstream(java.util.List extends
> org.apache.hadoop.hive.ql.plan.OperatorDesc>>,java.lang.Class)
> method
> org.apache.hadoop.hive.ql.exec.OperatorUtils.findOperatorsUpstream(org.apache.hadoop.hive.ql.exec.Operator,java.lang.Class)
> is not applicable
> (cannot infer type-variable(s) T
> (argument mismatch;
> java.util.List org.apache.hadoop.hive.ql.plan.OperatorDesc>> cannot be converted to
> org.apache.hadoop.hive.ql.exec.Operator))
> method
> org.apache.hadoop.hive.ql.exec.OperatorUtils.findOperatorsUpstream(java.util.Collection>,java.lang.Class)
> is not applicable
> (cannot infer type-variable(s) T
> (argument mismatch;
> java.util.List org.apache.hadoop.hive.ql.plan.OperatorDesc>> cannot be converted to
> java.util.Collection>))
> method
> org.apache.hadoop.hive.ql.exec.OperatorUtils.findOperatorsUpstream(org.apache.hadoop.hive.ql.exec.Operator,java.lang.Class,java.util.Set)
> is not applicable
> (cannot infer type-variable(s) T
> (actual and formal argument lists differ in length))
> {code}
> and
> {code}
> /Users/asherman/git/asf/hive2/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/AnnotateRunTimeStatsOptimizer.java
> Error:(76, 12) java: no suitable method found for
> addAll(java.util.List>)
> method java.util.Collection.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.List> cannot be
> converted to java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> method java.util.Set.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.List> cannot be
> converted to java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> Error:(80, 14) java: no suitable method found for
> addAll(java.util.Set>)
> method java.util.Collection.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.Set> cannot be converted
> to java.util.Collection extends
[jira] [Updated] (HIVE-19581) view do not support unicode characters well
[
https://issues.apache.org/jira/browse/HIVE-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-19581:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks [~ngangam] for the commit. My intention was to prevent future
regressions in master so I think this is good enough for mow.
> view do not support unicode characters well
> ---
>
> Key: HIVE-19581
> URL: https://issues.apache.org/jira/browse/HIVE-19581
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: kai
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-19581.1.patch, HIVE-19581.2.patch,
> HIVE-19581.3.patch, HIVE-19581.4.patch, HIVE-19581.5.patch,
> HIVE-19581.6.patch, explain.png, metastore.png
>
>
> create table t_test (name ,string) ;
> insert into table t_test VALUES ('李四');
> create view t_view_test as select * from t_test where name='李四';
> when select * from t_view_test no records return
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20030) Fix Java compile errors that show up in IntelliJ from ConvertJoinMapJoin.java and AnnotateRunTimeStatsOptimizer.java
[
https://issues.apache.org/jira/browse/HIVE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528838#comment-16528838
]
Andrew Sherman commented on HIVE-20030:
---
[~sershe] this could be user error (by me) but IntelliJ is using jdk8 and the
language level is 8.
The change does seem to make the types match more accurately so this seems like
a safe change.
> Fix Java compile errors that show up in IntelliJ from ConvertJoinMapJoin.java
> and AnnotateRunTimeStatsOptimizer.java
>
>
> Key: HIVE-20030
> URL: https://issues.apache.org/jira/browse/HIVE-20030
> Project: Hive
> Issue Type: Task
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-20030.1.patch
>
>
> For some reason the Java compiler in IntellJ is more strict that the Oracle
> jdk compiler. Maybe this is something that can be configured away, but as it
> is simple I propose to make the code more type correct.
> {code}
> /Users/asherman/git/asf/hive2/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java
> Error:(613, 24) java: no suitable method found for
> findOperatorsUpstream(java.util.List extends
> org.apache.hadoop.hive.ql.plan.OperatorDesc>>,java.lang.Class)
> method
> org.apache.hadoop.hive.ql.exec.OperatorUtils.findOperatorsUpstream(org.apache.hadoop.hive.ql.exec.Operator,java.lang.Class)
> is not applicable
> (cannot infer type-variable(s) T
> (argument mismatch;
> java.util.List org.apache.hadoop.hive.ql.plan.OperatorDesc>> cannot be converted to
> org.apache.hadoop.hive.ql.exec.Operator))
> method
> org.apache.hadoop.hive.ql.exec.OperatorUtils.findOperatorsUpstream(java.util.Collection>,java.lang.Class)
> is not applicable
> (cannot infer type-variable(s) T
> (argument mismatch;
> java.util.List org.apache.hadoop.hive.ql.plan.OperatorDesc>> cannot be converted to
> java.util.Collection>))
> method
> org.apache.hadoop.hive.ql.exec.OperatorUtils.findOperatorsUpstream(org.apache.hadoop.hive.ql.exec.Operator,java.lang.Class,java.util.Set)
> is not applicable
> (cannot infer type-variable(s) T
> (actual and formal argument lists differ in length))
> {code}
> and
> {code}
> /Users/asherman/git/asf/hive2/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/AnnotateRunTimeStatsOptimizer.java
> Error:(76, 12) java: no suitable method found for
> addAll(java.util.List>)
> method java.util.Collection.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.List> cannot be
> converted to java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> method java.util.Set.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.List> cannot be
> converted to java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> Error:(80, 14) java: no suitable method found for
> addAll(java.util.Set>)
> method java.util.Collection.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.Set> cannot be converted
> to java.util.Collection extends org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> method java.util.Set.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.Set> cannot be converted
> to java.util.Collection extends org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> Error:(85, 14) java: no suitable method found for
> addAll(java.util.Set>)
> method java.util.Collection.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.Set> cannot be converted
> to java.util.Collection extends org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> method java.util.Set.addAll(java.util.Collection org.apache.hadoop.hive.ql.exec.Operator org.apache.hadoop.hive.ql.plan.OperatorDesc>>) is not applicable
> (argument mismatch;
> java.util.Set> cannot be converted
> to java.util.Collection extends org.apache.hadoop.hive.ql.plan.OperatorDesc>>)
> /Users/asherman/git/asf/hive2/ql/target/generated-sources/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/IntervalYearMonthScalarAddTimestampColumn.java
> {code}
--
This message was sent by Atlassian
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: HIVE-18118.14.patch > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.11.patch, > HIVE-18118.12.patch, HIVE-18118.14.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, > HIVE-18118.6.patch, HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[ https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18118: -- Attachment: (was: HIVE-18118.13.patch) > Explain Extended should indicate if a file being read is an EC file > --- > > Key: HIVE-18118 > URL: https://issues.apache.org/jira/browse/HIVE-18118 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18118.1.patch, HIVE-18118.10.patch, > HIVE-18118.10.patch, HIVE-18118.11.patch, HIVE-18118.11.patch, > HIVE-18118.12.patch, HIVE-18118.14.patch, HIVE-18118.2.patch, > HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch, > HIVE-18118.6.patch, HIVE-18118.7.patch, HIVE-18118.8.patch, HIVE-18118.9.patch > > > We already print out the files Hive will read in the explain extended > command, we just have to modify it to say whether or not its an EC file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17826) Error writing to RandomAccessFile after operation log is closed
[
https://issues.apache.org/jira/browse/HIVE-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211942#comment-16211942
]
Andrew Sherman commented on HIVE-17826:
---
Thanks [~aihuaxu] I did think about that but I am not sure how to do it in a
simple and clean way. cleanupOperationLog() is called from Operation.closeO
so adding a delay inline will prevent the session from terminating which seems
weird. And doing it asynchronously makes it more complicated.
But as we just discussed IRL I will think about it some more.
> Error writing to RandomAccessFile after operation log is closed
> ---
>
> Key: HIVE-17826
> URL: https://issues.apache.org/jira/browse/HIVE-17826
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-17826.1.patch
>
>
> We are seeing the error from HS2 process stdout.
> {noformat}
> 2017-09-07 10:17:23,933 AsyncLogger-1 ERROR Attempted to append to
> non-started appender query-file-appender
> 2017-09-07 10:17:23,934 AsyncLogger-1 ERROR Attempted to append to
> non-started appender query-file-appender
> 2017-09-07 10:17:23,935 AsyncLogger-1 ERROR Unable to write to stream
> /var/log/hive/operation_logs/dd38df5b-3c09-48c9-ad64-a2eee093bea6/hive_20170907101723_1a6ad4b9-f662-4e7a-a495-06e3341308f9
> for appender query-file-appender
> 2017-09-07 10:17:23,935 AsyncLogger-1 ERROR An exception occurred processing
> Appender query-file-appender
> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error
> writing to RandomAccessFile
> /var/log/hive/operation_logs/dd38df5b-3c09-48c9-ad64-a2eee093bea6/hive_20170907101723_1a6ad4b9-f662-4e7a-a495-06e3341308f9
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.flush(RandomAccessFileManager.java:114)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.write(RandomAccessFileManager.java:103)
> at
> org.apache.logging.log4j.core.appender.OutputStreamManager.write(OutputStreamManager.java:136)
> at
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:105)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileAppender.append(RandomAccessFileAppender.java:89)
> at
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> at
> org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:112)
> at
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:390)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:378)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:362)
> at
> org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:79)
> at
> org.apache.logging.log4j.core.async.AsyncLogger.actualAsyncLog(AsyncLogger.java:385)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEvent.execute(RingBufferLogEvent.java:103)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:43)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:28)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Stream Closed
> at java.io.RandomAccessFile.writeBytes(Native Method)
> at java.io.RandomAccessFile.write(RandomAccessFile.java:525)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.flush(RandomAccessFileManager.java:111)
> ... 25 more
> {noformat}
[jira] [Commented] (HIVE-17826) Error writing to RandomAccessFile after operation log is closed
[
https://issues.apache.org/jira/browse/HIVE-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220661#comment-16220661
]
Andrew Sherman commented on HIVE-17826:
---
What do you think [~aihuaxu] ?
> Error writing to RandomAccessFile after operation log is closed
> ---
>
> Key: HIVE-17826
> URL: https://issues.apache.org/jira/browse/HIVE-17826
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-17826.1.patch
>
>
> We are seeing the error from HS2 process stdout.
> {noformat}
> 2017-09-07 10:17:23,933 AsyncLogger-1 ERROR Attempted to append to
> non-started appender query-file-appender
> 2017-09-07 10:17:23,934 AsyncLogger-1 ERROR Attempted to append to
> non-started appender query-file-appender
> 2017-09-07 10:17:23,935 AsyncLogger-1 ERROR Unable to write to stream
> /var/log/hive/operation_logs/dd38df5b-3c09-48c9-ad64-a2eee093bea6/hive_20170907101723_1a6ad4b9-f662-4e7a-a495-06e3341308f9
> for appender query-file-appender
> 2017-09-07 10:17:23,935 AsyncLogger-1 ERROR An exception occurred processing
> Appender query-file-appender
> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error
> writing to RandomAccessFile
> /var/log/hive/operation_logs/dd38df5b-3c09-48c9-ad64-a2eee093bea6/hive_20170907101723_1a6ad4b9-f662-4e7a-a495-06e3341308f9
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.flush(RandomAccessFileManager.java:114)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.write(RandomAccessFileManager.java:103)
> at
> org.apache.logging.log4j.core.appender.OutputStreamManager.write(OutputStreamManager.java:136)
> at
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:105)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileAppender.append(RandomAccessFileAppender.java:89)
> at
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> at
> org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:112)
> at
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:390)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:378)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:362)
> at
> org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:79)
> at
> org.apache.logging.log4j.core.async.AsyncLogger.actualAsyncLog(AsyncLogger.java:385)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEvent.execute(RingBufferLogEvent.java:103)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:43)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:28)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Stream Closed
> at java.io.RandomAccessFile.writeBytes(Native Method)
> at java.io.RandomAccessFile.write(RandomAccessFile.java:525)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.flush(RandomAccessFileManager.java:111)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17868) Make queries in spark_local_queries.q have deterministic output
[ https://issues.apache.org/jira/browse/HIVE-17868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17868: -- Attachment: HIVE-17868.1.patch > Make queries in spark_local_queries.q have deterministic output > --- > > Key: HIVE-17868 > URL: https://issues.apache.org/jira/browse/HIVE-17868 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-17868.1.patch > > > Add 'order by' to queries so that output is always the same -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17868) Make queries in spark_local_queries.q have deterministic output
[ https://issues.apache.org/jira/browse/HIVE-17868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17868: -- Status: Patch Available (was: Open) This is a small change > Make queries in spark_local_queries.q have deterministic output > --- > > Key: HIVE-17868 > URL: https://issues.apache.org/jira/browse/HIVE-17868 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-17868.1.patch > > > Add 'order by' to queries so that output is always the same -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17635) Add unit tests to CompactionTxnHandler and use PreparedStatements for queries
[ https://issues.apache.org/jira/browse/HIVE-17635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215468#comment-16215468 ] Andrew Sherman commented on HIVE-17635: --- Test failures are unconnected with this change. So this is ready to push, can you take a look [~stakiar] ? > Add unit tests to CompactionTxnHandler and use PreparedStatements for queries > - > > Key: HIVE-17635 > URL: https://issues.apache.org/jira/browse/HIVE-17635 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-17635.1.patch, HIVE-17635.2.patch, > HIVE-17635.3.patch, HIVE-17635.4.patch, HIVE-17635.6.patch > > > It is better for jdbc code that runs against the HMS database to use > PreparedStatements. Convert CompactionTxnHandler queries to use > PreparedStatement and add tests to TestCompactionTxnHandler to test these > queries, and improve code coverage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17826) Error writing to RandomAccessFile after operation log is closed
[
https://issues.apache.org/jira/browse/HIVE-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222677#comment-16222677
]
Andrew Sherman commented on HIVE-17826:
---
Thanks [~aihuaxu], do we need more review? If not can you push this when
convenient?
> Error writing to RandomAccessFile after operation log is closed
> ---
>
> Key: HIVE-17826
> URL: https://issues.apache.org/jira/browse/HIVE-17826
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-17826.1.patch
>
>
> We are seeing the error from HS2 process stdout.
> {noformat}
> 2017-09-07 10:17:23,933 AsyncLogger-1 ERROR Attempted to append to
> non-started appender query-file-appender
> 2017-09-07 10:17:23,934 AsyncLogger-1 ERROR Attempted to append to
> non-started appender query-file-appender
> 2017-09-07 10:17:23,935 AsyncLogger-1 ERROR Unable to write to stream
> /var/log/hive/operation_logs/dd38df5b-3c09-48c9-ad64-a2eee093bea6/hive_20170907101723_1a6ad4b9-f662-4e7a-a495-06e3341308f9
> for appender query-file-appender
> 2017-09-07 10:17:23,935 AsyncLogger-1 ERROR An exception occurred processing
> Appender query-file-appender
> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error
> writing to RandomAccessFile
> /var/log/hive/operation_logs/dd38df5b-3c09-48c9-ad64-a2eee093bea6/hive_20170907101723_1a6ad4b9-f662-4e7a-a495-06e3341308f9
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.flush(RandomAccessFileManager.java:114)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.write(RandomAccessFileManager.java:103)
> at
> org.apache.logging.log4j.core.appender.OutputStreamManager.write(OutputStreamManager.java:136)
> at
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:105)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileAppender.append(RandomAccessFileAppender.java:89)
> at
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> at
> org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:112)
> at
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> at
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:390)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:378)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:362)
> at
> org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:79)
> at
> org.apache.logging.log4j.core.async.AsyncLogger.actualAsyncLog(AsyncLogger.java:385)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEvent.execute(RingBufferLogEvent.java:103)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:43)
> at
> org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:28)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Stream Closed
> at java.io.RandomAccessFile.writeBytes(Native Method)
> at java.io.RandomAccessFile.write(RandomAccessFile.java:525)
> at
> org.apache.logging.log4j.core.appender.RandomAccessFileManager.flush(RandomAccessFileManager.java:111)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18852) Misleading error message in alter table validation
[
https://issues.apache.org/jira/browse/HIVE-18852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned HIVE-18852:
-
Assignee: Andrew Sherman
> Misleading error message in alter table validation
> --
>
> Key: HIVE-18852
> URL: https://issues.apache.org/jira/browse/HIVE-18852
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.4.0
>Reporter: Dan Burkert
>Assignee: Andrew Sherman
>Priority: Major
>
> The metastore's validation error message when attempting to rename a table to
> a non-existent database is wrong. For instance, attempting to alter table
> 'db.table' to 'non_existent_database.table' results in the Thrift error:
> {{TException - service has thrown: InvalidOperationException(message=Unable
> to change partition or table. Database db does not exist Check metastore logs
> for detailed stack.non_existent_database)}}
> I believe the offending line of code is
> [here|https://github.com/apache/hive/blob/branch-2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java?utf8=%E2%9C%93#L331-L333],
> notice that {{dbname}} is used in the message, not {{newDbName}}. I don't
> know if switching that would cause the case of a non-existing {{dbname}} case
> to regress, though.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18852) Misleading error message in alter table validation
[
https://issues.apache.org/jira/browse/HIVE-18852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-18852:
--
Attachment: HIVE-18852.1.patch
Status: Patch Available (was: Open)
> Misleading error message in alter table validation
> --
>
> Key: HIVE-18852
> URL: https://issues.apache.org/jira/browse/HIVE-18852
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.4.0
>Reporter: Dan Burkert
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-18852.1.patch
>
>
> The metastore's validation error message when attempting to rename a table to
> a non-existent database is wrong. For instance, attempting to alter table
> 'db.table' to 'non_existent_database.table' results in the Thrift error:
> {{TException - service has thrown: InvalidOperationException(message=Unable
> to change partition or table. Database db does not exist Check metastore logs
> for detailed stack.non_existent_database)}}
> I believe the offending line of code is
> [here|https://github.com/apache/hive/blob/branch-2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java?utf8=%E2%9C%93#L331-L333],
> notice that {{dbname}} is used in the message, not {{newDbName}}. I don't
> know if switching that would cause the case of a non-existing {{dbname}} case
> to regress, though.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-14615) Temp table leaves behind insert command
[
https://issues.apache.org/jira/browse/HIVE-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304695#comment-16304695
]
Andrew Sherman commented on HIVE-14615:
---
Hi [~minions] I actually have 2 different fixes for this all coded up. I have
been distracted for a while and have not submitted. Did you have an idea of how
to fix this?
> Temp table leaves behind insert command
> ---
>
> Key: HIVE-14615
> URL: https://issues.apache.org/jira/browse/HIVE-14615
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Chaoyu Tang
>Assignee: Madhudeep Petwal
>
> {code}
> create table test (key int, value string);
> insert into test values (1, 'val1');
> show tables;
> test
> values__tmp__table__1
> {code}
> the temp table values__tmp__table__1 was resulted from insert into ...values
> and exists until logout the session.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18336) add Safe Mode
[ https://issues.apache.org/jira/browse/HIVE-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304752#comment-16304752 ] Andrew Sherman commented on HIVE-18336: --- [~ekoifman] Can you explain a bit more what this jira means? Thanks > add Safe Mode > - > > Key: HIVE-18336 > URL: https://issues.apache.org/jira/browse/HIVE-18336 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-14615) Temp table leaves behind insert command
[
https://issues.apache.org/jira/browse/HIVE-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-14615:
--
Attachment: HIVE-14615.1.patch
> Temp table leaves behind insert command
> ---
>
> Key: HIVE-14615
> URL: https://issues.apache.org/jira/browse/HIVE-14615
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Chaoyu Tang
>Assignee: Andrew Sherman
> Attachments: HIVE-14615.1.patch
>
>
> {code}
> create table test (key int, value string);
> insert into test values (1, 'val1');
> show tables;
> test
> values__tmp__table__1
> {code}
> the temp table values__tmp__table__1 was resulted from insert into ...values
> and exists until logout the session.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-14615) Temp table leaves behind insert command
[
https://issues.apache.org/jira/browse/HIVE-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-14615:
--
Status: Patch Available (was: Open)
> Temp table leaves behind insert command
> ---
>
> Key: HIVE-14615
> URL: https://issues.apache.org/jira/browse/HIVE-14615
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Chaoyu Tang
>Assignee: Andrew Sherman
> Attachments: HIVE-14615.1.patch
>
>
> {code}
> create table test (key int, value string);
> insert into test values (1, 'val1');
> show tables;
> test
> values__tmp__table__1
> {code}
> the temp table values__tmp__table__1 was resulted from insert into ...values
> and exists until logout the session.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-14615) Temp table leaves behind insert command
[
https://issues.apache.org/jira/browse/HIVE-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-14615:
--
Attachment: HIVE-14615.4.patch
need to rebase :-(
> Temp table leaves behind insert command
> ---
>
> Key: HIVE-14615
> URL: https://issues.apache.org/jira/browse/HIVE-14615
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Chaoyu Tang
>Assignee: Andrew Sherman
> Attachments: HIVE-14615.1.patch, HIVE-14615.2.patch,
> HIVE-14615.3.patch, HIVE-14615.4.patch
>
>
> {code}
> create table test (key int, value string);
> insert into test values (1, 'val1');
> show tables;
> test
> values__tmp__table__1
> {code}
> the temp table values__tmp__table__1 was resulted from insert into ...values
> and exists until logout the session.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312271#comment-16312271 ] Andrew Sherman commented on HIVE-18367: --- Test failures look unrelated to this change. [~pvary] please could you review? > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18367.1.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-14615) Temp table leaves behind insert command
[
https://issues.apache.org/jira/browse/HIVE-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-14615:
--
Attachment: HIVE-14615.3.patch
> Temp table leaves behind insert command
> ---
>
> Key: HIVE-14615
> URL: https://issues.apache.org/jira/browse/HIVE-14615
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Chaoyu Tang
>Assignee: Andrew Sherman
> Attachments: HIVE-14615.1.patch, HIVE-14615.2.patch,
> HIVE-14615.3.patch
>
>
> {code}
> create table test (key int, value string);
> insert into test values (1, 'val1');
> show tables;
> test
> values__tmp__table__1
> {code}
> the temp table values__tmp__table__1 was resulted from insert into ...values
> and exists until logout the session.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18375) Cannot ORDER by subquery fields unless they are selected
[
https://issues.apache.org/jira/browse/HIVE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311995#comment-16311995
]
Andrew Sherman commented on HIVE-18375:
---
Great bug report.
[~minions] do you want to take a look at this?
> Cannot ORDER by subquery fields unless they are selected
>
>
> Key: HIVE-18375
> URL: https://issues.apache.org/jira/browse/HIVE-18375
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.3.2
> Environment: Amazon AWS
> Release label:emr-5.11.0
> Hadoop distribution:Amazon 2.7.3
> Applications:Hive 2.3.2, Pig 0.17.0, Hue 4.0.1
> classification=hive-site,properties=[hive.strict.checks.cartesian.product=false,hive.mapred.mode=nonstrict]
>Reporter: Paul Jackson
>Priority: Minor
>
> Give these tables:
> {code:SQL}
> CREATE TABLE employees (
> emp_no INT,
> first_name VARCHAR(14),
> last_name VARCHAR(16)
> );
> insert into employees values
> (1, 'Gottlob', 'Frege'),
> (2, 'Bertrand', 'Russell'),
> (3, 'Ludwig', 'Wittgenstein');
> CREATE TABLE salaries (
> emp_no INT,
> salary INT,
> from_date DATE,
> to_date DATE
> );
> insert into salaries values
> (1, 10, '1900-01-01', '1900-01-31'),
> (1, 18, '1900-09-01', '1900-09-30'),
> (2, 15, '1940-03-01', '1950-01-01'),
> (3, 20, '1920-01-01', '1950-01-01');
> {code}
> This query returns the names of the employees ordered by their peak salary:
> {code:SQL}
> SELECT `employees`.`last_name`, `employees`.`first_name`, `t1`.`max_salary`
> FROM `default`.`employees`
> INNER JOIN
> (SELECT `emp_no`, MAX(`salary`) `max_salary`
> FROM `default`.`salaries`
> WHERE `emp_no` IS NOT NULL AND `salary` IS NOT NULL
> GROUP BY `emp_no`) AS `t1`
> ON `employees`.`emp_no` = `t1`.`emp_no`
> ORDER BY `t1`.`max_salary` DESC;
> {code}
> However, this should still work even if the max_salary is not part of the
> projection:
> {code:SQL}
> SELECT `employees`.`last_name`, `employees`.`first_name`
> FROM `default`.`employees`
> INNER JOIN
> (SELECT `emp_no`, MAX(`salary`) `max_salary`
> FROM `default`.`salaries`
> WHERE `emp_no` IS NOT NULL AND `salary` IS NOT NULL
> GROUP BY `emp_no`) AS `t1`
> ON `employees`.`emp_no` = `t1`.`emp_no`
> ORDER BY `t1`.`max_salary` DESC;
> {code}
> However, that fails with this error:
> {code}
> Error while compiling statement: FAILED: SemanticException [Error 10004]:
> line 9:9 Invalid table alias or column reference 't1': (possible column names
> are: last_name, first_name)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18367: -- Attachment: HIVE-18367.2.patch HIVE-18367.2.patch adds a BeeLine test and uses HiveStringUtils.escapeJava(). I confirmed that CliDriver output is the same by hand - the output is masked from a .q file test. > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[
https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16319015#comment-16319015
]
Andrew Sherman commented on HIVE-18367:
---
Old splits Detailed Table Information if newlines are present in view
definition:
{noformat}
Detailed Table Information Table(tableName:view9, dbName:default,
owner:hive_test_user, createTime:1515524236, lastAccessTime:0, retention:0,
sd:StorageDescriptor(cols:[FieldSchema(name:m, type:int, comment:null)],
location:null, inputFormat:org.apache.hadoop.mapred.TextInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null,
serializationLib:null, parameters:{}), bucketCols:[], sortCols:[],
parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[],
skewedColValueLocationMaps:{}), storedAsSubDirectories:false),
partitionKeys:[], parameters:{transient_lastDdlTime=1515524236},
viewOriginalText:SELECT test_max(length(value))
FROM src, viewExpandedText:SELECT `_c0` AS `m` FROM (SELECT
`test_max`(length(`src`.`value`))
FROM `default`.`src`) `default.view9`, tableType:VIRTUAL_VIEW,
rewriteEnabled:false)
{noformat}
New adds \n instead:
{noformat}
Detailed Table Information Table(tableName:view9, dbName:default,
owner:hive_test_user, createTime:1515526192, lastAccessTime:0, retention:0,
sd:StorageDescriptor(cols:[FieldSchema(name:m, type:int, comment:null)],
location:null, inputFormat:org.apache.hadoop.mapred.TextInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null,
serializationLib:null, parameters:{}), bucketCols:[], sortCols:[],
parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[],
skewedColValueLocationMaps:{}), storedAsSubDirectories:false),
partitionKeys:[], parameters:{transient_lastDdlTime=1515526192},
viewOriginalText:SELECT test_max(length(value))\nFROM src,
viewExpandedText:SELECT `_c0` AS `m` FROM (SELECT
`test_max`(length(`src`.`value`))\nFROM `default`.`src`) `default.view9`,
tableType:VIRTUAL_VIEW, rewriteEnabled:false)
{noformat}
Note that exact view text from DESCRIBE FORMATTED remains unchanged.
Do you think this is OK [~pvary] ?
> Describe Extended output is truncated on a table with an explicit row format
> containing tabs or newlines.
> -
>
> Key: HIVE-18367
> URL: https://issues.apache.org/jira/browse/HIVE-18367
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch,
> HIVE-18367.3.patch
>
>
> 'Describe Extended' dumps information about a table. The protocol for sending
> this data relies on tabs and newlines to separate pieces of data. If a table
> has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a
> tab or newline then the output seen by the user is prematurely truncated. Fix
> this by replacing tabs and newlines in the table description with “\n” and
> “\t”.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316655#comment-16316655 ] Andrew Sherman commented on HIVE-17952: --- [~prasanth_j] any chance you could push this change please? > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18367: -- Attachment: HIVE-18367.3.patch Some test changes because the 'describe extended' output now always comes on one line. This seems better. Anyone who needs to parse the output (for example to get view text) can use 'describe formatted' > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311657#comment-16311657 ] Andrew Sherman commented on HIVE-17952: --- Test failures look unrelated. [~pvary] and [~prasanth_j] please review. > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18346) Beeline could not launch because of size of history
[
https://issues.apache.org/jira/browse/HIVE-18346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311617#comment-16311617
]
Andrew Sherman commented on HIVE-18346:
---
Hi [~minions] I'm a beginner myself. I would suggest (1) looking at [unassigned
hive
bugs|https://issues.apache.org/jira/browse/HIVE-18371?jql=project%20%3D%20HIVE%20AND%20assignee%20in%20(EMPTY)%20ORDER%20BY%20created%20DESC]
(2) using the product and finding your own problems. The beginner sees things
that the more experienced people miss!
You should also ask on the dev mailing list for suggestions.
> Beeline could not launch because of size of history
> ---
>
> Key: HIVE-18346
> URL: https://issues.apache.org/jira/browse/HIVE-18346
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Jumping
>Assignee: Madhudeep Petwal
>Priority: Minor
>
> Beeline version 1.2.1 could not launch when the size of
> ${user.home}/.beeline/history larger than 39MB. Which reports
> "java.lang.outofmemoryerror" .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311669#comment-16311669 ] Andrew Sherman commented on HIVE-18367: --- [~pvary] yes [HIVE-14146|https://issues.apache.org/jira/browse/HIVE-14146] is in exactly the same area. This is another variant. The whole area seems soft. > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18367.1.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18419) CliDriver loads different hive-site.xml into HiveConf and MetastoreConf
[
https://issues.apache.org/jira/browse/HIVE-18419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321373#comment-16321373
]
Andrew Sherman commented on HIVE-18419:
---
Is it possible to write a test for this?
> CliDriver loads different hive-site.xml into HiveConf and MetastoreConf
> ---
>
> Key: HIVE-18419
> URL: https://issues.apache.org/jira/browse/HIVE-18419
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore, Test
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-18419.01.patch
>
>
> Various forms of CliDriver use CliConfigs to set the 'confDir' below (e.g.
> CliConfigs.SparkOnYarnCliConfig() used by TestMiniSparkOnYarnCliDriver.
> QTestUtil.QTestUtil() has
> {noformat}
> if (confDir != null && !confDir.isEmpty()) {
> HiveConf.setHiveSiteLocation(new URL("file://"+ new
> File(confDir).toURI().getPath() + "/hive-site.xml"));
> MetastoreConf.setHiveSiteLocation(HiveConf.getHiveSiteLocation());
> System.out.println("Setting hive-site:
> "+HiveConf.getHiveSiteLocation());
> }
> {noformat}
> This causes HiveConf.initialize() to load hvie-site.xml from that location.
> MetastoreConf only loads hive-site.xml from the classpath which in the test
> environment picks up data/conf/hive-site.xml
> So different parts of the system may end up disagreeing about property values.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327932#comment-16327932 ] Andrew Sherman commented on HIVE-17952: --- [~prasanth_j] could you please review and push this change? > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327944#comment-16327944 ] Andrew Sherman commented on HIVE-17952: --- [~prasanth_j] thanks, will do rebase today or tomorrow > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18367: -- Attachment: HIVE-18367.5.patch > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch, HIVE-18367.4.patch, HIVE-18367.5.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17952: -- Attachment: HIVE-17952.4.patch > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch, HIVE-17952.2.patch, > HIVE-17952.3.patch, HIVE-17952.4.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18367: -- Attachment: HIVE-18367.6.patch > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch, HIVE-18367.4.patch, HIVE-18367.5.patch, HIVE-18367.6.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18367: -- Attachment: HIVE-18367.4.patch > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch, HIVE-18367.4.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327371#comment-16327371 ] Andrew Sherman commented on HIVE-18367: --- Created HIVE-18456 for followup tests > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch, HIVE-18367.4.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned HIVE-18456: - > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332624#comment-16332624 ] Andrew Sherman commented on HIVE-17952: --- [~prasanth_j] I finally got tests to run. Failures are unconnected with this change. So this change is ready to commit. > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch, HIVE-17952.2.patch, > HIVE-17952.3.patch, HIVE-17952.4.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18471) Beeline gives log4j warnings
[ https://issues.apache.org/jira/browse/HIVE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333028#comment-16333028 ] Andrew Sherman commented on HIVE-18471: --- [~janulatha] did you create 2 bugs by mistake? Is this a duplicate of HIVE-18472 > Beeline gives log4j warnings > > > Key: HIVE-18471 > URL: https://issues.apache.org/jira/browse/HIVE-18471 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Janaki Lahorani >Priority: Major > > Starting Beeline gives the following warnings: > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an > explanation. > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] > ERROR StatusLogger No log4j2 configuration file found. Using default > configuration: logging only errors to the console. Set system property > 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show > Log4j2 internal initialization logging. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333018#comment-16333018 ] Andrew Sherman commented on HIVE-17952: --- Thanks [~prasanth_j] > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-17952.1.patch, HIVE-17952.2.patch, > HIVE-17952.3.patch, HIVE-17952.4.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18495) JUnit rule to enable Driver level testing
[
https://issues.apache.org/jira/browse/HIVE-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333033#comment-16333033
]
Andrew Sherman commented on HIVE-18495:
---
[~kgyrtkirk] this looks interesting, can you explain a bit more about how to
use it?
> JUnit rule to enable Driver level testing
> -
>
> Key: HIVE-18495
> URL: https://issues.apache.org/jira/browse/HIVE-18495
> Project: Hive
> Issue Type: Sub-task
> Components: Testing Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18495.01.patch
>
>
> I've tried to write a case for a sophisticated check...it worked so well that
> I've started using it and eventually created a junit rule to make it easier
> to reuse
> Currently it takes ~15-25sec to run a test case with this framework (from
> which most of the time is the launch time of all kind of stuff which are
> needed to run a driver command).
> * enable to write JUnit tests which has access to the {{IDriver}} level
> * leave out the cli-driver; it sometimes causes problems
> * write tests at the {{ql}} module
> * it should also work from the IDE without changing anything
> Note: JUnit 5 would be great for this task; but unfortunately junit5 needs
> maven-surefire 2.19.1 ; which causes all kinds of problems for hive devs
> using idea...so that's not an option.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333055#comment-16333055 ] Andrew Sherman commented on HIVE-18367: --- Finally I got tests to run. Failures are unconnected with this change. [~pvary] please can you commit if possible? > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch, HIVE-18367.4.patch, HIVE-18367.5.patch, HIVE-18367.6.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334657#comment-16334657 ] Andrew Sherman commented on HIVE-18367: --- Than you [~pvary] > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch, HIVE-18367.4.patch, HIVE-18367.5.patch, HIVE-18367.6.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17952: -- Attachment: HIVE-17952.2.patch > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch, HIVE-17952.2.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17952: -- Attachment: HIVE-17952.3.patch > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch, HIVE-17952.2.patch, > HIVE-17952.3.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17935) Turn on hive.optimize.sort.dynamic.partition by default
[ https://issues.apache.org/jira/browse/HIVE-17935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17935: -- Attachment: HIVE-17935.8.patch > Turn on hive.optimize.sort.dynamic.partition by default > --- > > Key: HIVE-17935 > URL: https://issues.apache.org/jira/browse/HIVE-17935 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-17935.1.patch, HIVE-17935.2.patch, > HIVE-17935.3.patch, HIVE-17935.4.patch, HIVE-17935.5.patch, > HIVE-17935.6.patch, HIVE-17935.7.patch, HIVE-17935.8.patch > > > The config option hive.optimize.sort.dynamic.partition is an optimization for > Hive’s dynamic partitioning feature. It was originally implemented in > [HIVE-6455|https://issues.apache.org/jira/browse/HIVE-6455]. With this > optimization, the dynamic partition columns and bucketing columns (in case of > bucketed tables) are sorted before being fed to the reducers. Since the > partitioning and bucketing columns are sorted, each reducer can keep only one > record writer open at any time thereby reducing the memory pressure on the > reducers. There were some early problems with this optimization and it was > disabled by default in HiveConf in > [HIVE-8151|https://issues.apache.org/jira/browse/HIVE-8151]. Since then > setting hive.optimize.sort.dynamic.partition=true has been used to solve > problems where dynamic partitioning produces with (1) too many small files on > HDFS, which is bad for the cluster and can increase overhead for future Hive > queries over those partitions, and (2) OOM issues in the map tasks because it > trying to simultaneously write to 100 different files. > It now seems that the feature is probably mature enough that it can be > enabled by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-18625:
--
Attachment: HIVE-18625.1.patch
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-18625:
--
Status: Patch Available (was: Open)
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 2.3.2, 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-18625:
--
Attachment: (was: HIVE-18625.2.patch)
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-18625:
--
Attachment: HIVE-18625.2.patch
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch, HIVE-18625.2.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18456: -- Attachment: HIVE-18456.2.patch > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch, HIVE-18456.2.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-18625:
--
Attachment: HIVE-18625.2.patch
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch, HIVE-18625.2.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356105#comment-16356105
]
Andrew Sherman commented on HIVE-18625:
---
{color:#33}Test failures appear unrelated.{color}
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18456: -- Status: Patch Available (was: Open) > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18456: -- Attachment: HIVE-18456.1.patch > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358886#comment-16358886
]
Andrew Sherman commented on HIVE-18625:
---
Made changes, ran tests, and failures look unrelated. Please can you take
another look [~stakiar]
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch, HIVE-18625.2.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18456: -- Attachment: HIVE-18456.3.patch > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch, HIVE-18456.2.patch, > HIVE-18456.3.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361662#comment-16361662 ] Andrew Sherman commented on HIVE-18456: --- Tidied up per [~pvary] suggestions. Test failures and ASF License warnings are not caused by this change. > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch, HIVE-18456.2.patch, > HIVE-18456.3.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358893#comment-16358893 ] Andrew Sherman commented on HIVE-18456: --- Test failures look unrelated (not surprisingly as this is a test-only change). Please can you take a look [~pvary] > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch, HIVE-18456.2.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned HIVE-18625:
-
Assignee: Andrew Sherman
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370381#comment-16370381 ] Andrew Sherman commented on HIVE-18456: --- [~pvary] is there any more review needed? Otherwise please can you push this to master? > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch, HIVE-18456.2.patch, > HIVE-18456.3.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18791) Fix TestJdbcWithMiniHS2#testHttpHeaderSize
[ https://issues.apache.org/jira/browse/HIVE-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned HIVE-18791: - > Fix TestJdbcWithMiniHS2#testHttpHeaderSize > -- > > Key: HIVE-18791 > URL: https://issues.apache.org/jira/browse/HIVE-18791 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > TestJdbcWithMiniHS2#testHttpHeaderSize tests whether config of http header > sizes works by using a long username. The local scratch directory for the > session uses the username as part of its path. When this name is more than > 255 chars (on most modern file systems) then the directory creation will > fail. HIVE-18625 made this failure throw an exception, which has caused a > regression in testHttpHeaderSize. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
