[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477665#comment-13477665 ] Carl Steinbach commented on HIVE-2935: -- We're not planning to combine these services. Rather, we're planning to run these services in parallel in the same JVM, but each service will operate on its own unique port. If you want to use the HS2 API and the metastore API at the same time you will need to create two separate connections and use two separate clients. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3589) describe command not giving the details when called as describe dbname.tablename
[ https://issues.apache.org/jira/browse/HIVE-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3589: - Component/s: (was: CLI) Query Processor Metastore describe command not giving the details when called as describe dbname.tablename Key: HIVE-3589 URL: https://issues.apache.org/jira/browse/HIVE-3589 Project: Hive Issue Type: Bug Components: Metastore, Query Processor Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Priority: Minor describe command not giving the details when called as describe dbname.tablename. Throwing the error Table dbname not found. Ex: hive -e describe masterdb.table1 will throw error Table masterdb not found -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-967) Implement show create table
[ https://issues.apache.org/jira/browse/HIVE-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477711#comment-13477711 ] Hudson commented on HIVE-967: - Integrated in Hive-trunk-h0.21 #1742 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1742/]) HIVE-967. Implement show create table (Feng Lu via kevinwilfong) (Revision 1398896) Result = FAILURE kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1398896 Files : * /hive/trunk/build.xml * /hive/trunk/eclipse-templates/.classpath * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ShowCreateTableDesc.java * /hive/trunk/ql/src/test/queries/clientnegative/show_create_table_does_not_exist.q * /hive/trunk/ql/src/test/queries/clientnegative/show_create_table_index.q * /hive/trunk/ql/src/test/queries/clientpositive/show_create_table_alter.q * /hive/trunk/ql/src/test/queries/clientpositive/show_create_table_db_table.q * /hive/trunk/ql/src/test/queries/clientpositive/show_create_table_delimited.q * /hive/trunk/ql/src/test/queries/clientpositive/show_create_table_partitioned.q * /hive/trunk/ql/src/test/queries/clientpositive/show_create_table_serde.q * /hive/trunk/ql/src/test/queries/clientpositive/show_create_table_view.q * /hive/trunk/ql/src/test/results/clientnegative/show_create_table_does_not_exist.q.out * /hive/trunk/ql/src/test/results/clientnegative/show_create_table_index.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_create_table_alter.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_create_table_db_table.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_create_table_delimited.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_create_table_partitioned.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_create_table_serde.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_create_table_view.q.out Implement show create table - Key: HIVE-967 URL: https://issues.apache.org/jira/browse/HIVE-967 Project: Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Adam Kramer Assignee: Feng Lu Fix For: 0.10.0 Attachments: HIVE-967.2.patch.txt, HIVE-967.3.patch.txt, HIVE-967.4.patch.txt, HIVE-967.5.patch.txt, HIVE-967.6.patch.txt, HIVE-967.patch.txt, HiveShowCreateTable.jar, show_create.txt SHOW CREATE TABLE would be very useful in cases where you are trying to figure out the partitioning and/or bucketing scheme for a table. Perhaps this could be implemented by having new tables automatically SET PROPERTIES (create_command='raw text of the create statement')? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1742 - Still Failing
Changes for Build #1739 Changes for Build #1740 [namit] HIVE-3581 get_json_object and json_tuple return null in the presence of new line characters (Kevin Wilfong via namit) Changes for Build #1741 Changes for Build #1742 [kevinwilfong] HIVE-967. Implement show create table (Feng Lu via kevinwilfong) 1 tests failed. FAILED: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:11319) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1742) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1742/ to view the results.
[jira] [Assigned] (HIVE-3582) NPE in union processing followed by lateral view followed by 2 group bys
[ https://issues.apache.org/jira/browse/HIVE-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis reassigned HIVE-3582: --- Assignee: Navis NPE in union processing followed by lateral view followed by 2 group bys Key: HIVE-3582 URL: https://issues.apache.org/jira/browse/HIVE-3582 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Navis EXPLAIN SELECT e.key, e.arr_ele, count(1) FROM ( SELECT d.key as key, d.arr_ele as arr_ele, d.value as value, count(1) as cnt FROM ( SELECT c.arr_ele as arr_ele, a.key as key, a.value as value FROM ( SELECT key, value, array(1,2,3) as arr FROM src UNION ALL SELECT key, value, array(1,2,3) as arr FROM srcpart WHERE ds = '2008-04-08' and hr='12' ) a LATERAL VIEW EXPLODE(arr) c AS arr_ele ) d group by d.key, d.arr_ele, d.value ) e group by e.key, e.arr_ele; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
Esteban Gutierrez created HIVE-3590:
---
Summary: TCP KeepAlive and connection timeout for the HiveServer
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Fix For: 0.10.0
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer in
order to avoid half open connections or connection drops during long running
queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HIVE-3590:
Attachment: HIVE-3590.1.patch
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HIVE-3590:
Labels: patch (was: )
Status: Patch Available (was: Open)
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477718#comment-13477718
]
Esteban Gutierrez commented on HIVE-3590:
-
Review Board:
https://reviews.apache.org/r/7634/
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: TCP KeepAlive and connection timeout for the HiveServer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7634/ --- Review request for hive. Description --- Simple implementation of TCP KeepAlive and socket timeout for the HiveServer in order to avoid half open connections or connection drops during long running queries that use HiveServer. Two new settings have been added in order to control this functionality: hive.server.read.socket.timeout and hive.server.tcp.keepalive the proposal is to have hive.server.tcp.keepalive set by default to true. Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f86d6a7 conf/hive-default.xml.template 4a59fb6 service/src/java/org/apache/hadoop/hive/service/HiveServer.java 2b2f681 Diff: https://reviews.apache.org/r/7634/diff/ Testing --- Thanks, Esteban Gutierrez
Re: Review Request: HIVE-3590: TCP KeepAlive and connection timeout for the HiveServer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7634/ --- (Updated Oct. 17, 2012, 8:58 a.m.) Review request for hive. Summary (updated) - HIVE-3590: TCP KeepAlive and connection timeout for the HiveServer Description --- Simple implementation of TCP KeepAlive and socket timeout for the HiveServer in order to avoid half open connections or connection drops during long running queries that use HiveServer. Two new settings have been added in order to control this functionality: hive.server.read.socket.timeout and hive.server.tcp.keepalive the proposal is to have hive.server.tcp.keepalive set by default to true. Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f86d6a7 conf/hive-default.xml.template 4a59fb6 service/src/java/org/apache/hadoop/hive/service/HiveServer.java 2b2f681 Diff: https://reviews.apache.org/r/7634/diff/ Testing --- Thanks, Esteban Gutierrez
Re: Review Request: HIVE-3590: TCP KeepAlive and connection timeout for the HiveServer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7634/#review12510 --- Looks good. Needs one minor edit to the property description. conf/hive-default.xml.template https://reviews.apache.org/r/7634/#comment26641 s/defaults is /defaults to/ - Carl Steinbach On Oct. 17, 2012, 8:58 a.m., Esteban Gutierrez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7634/ --- (Updated Oct. 17, 2012, 8:58 a.m.) Review request for hive. Description --- Simple implementation of TCP KeepAlive and socket timeout for the HiveServer in order to avoid half open connections or connection drops during long running queries that use HiveServer. Two new settings have been added in order to control this functionality: hive.server.read.socket.timeout and hive.server.tcp.keepalive the proposal is to have hive.server.tcp.keepalive set by default to true. Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f86d6a7 conf/hive-default.xml.template 4a59fb6 service/src/java/org/apache/hadoop/hive/service/HiveServer.java 2b2f681 Diff: https://reviews.apache.org/r/7634/diff/ Testing --- Thanks, Esteban Gutierrez
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-3590:
-
Status: Open (was: Patch Available)
@Esteban: I left comments for you on reviewboard. Tnanks.
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HIVE-3590:
Attachment: HIVE-3590.2.patch
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch, HIVE-3590.2.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477734#comment-13477734
]
Esteban Gutierrez commented on HIVE-3590:
-
@Carl Updated the patch with the suggested change. Thanks.
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch, HIVE-3590.2.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HIVE-3590:
Status: Patch Available (was: Open)
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch, HIVE-3590.2.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HIVE-3590:
Description:
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer in
order to avoid half open connections or connection drops during long running
queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults to 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
was:
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer in
order to avoid half open connections or connection drops during long running
queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch, HIVE-3590.2.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults to 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477735#comment-13477735
]
Carl Steinbach commented on HIVE-3590:
--
+1. Will commit in a day if tests pass and no one requests more time for review.
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch, HIVE-3590.2.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults to 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3581) get_json_object and json_tuple return null in the presence of new line characters
[ https://issues.apache.org/jira/browse/HIVE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477800#comment-13477800 ] Philip Tromans commented on HIVE-3581: -- This patch breaks hive trunk on Hadoop 0.20.205.0, I guess because of the difference in Jackson versions. Running hive gives: Exception in thread main java.lang.NoSuchFieldError: ALLOW_UNQUOTED_CONTROL_CHARS at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.clinit(GenericUDTFJSONTuple.java:59) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:545) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:539) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.clinit(FunctionRegistry.java:472) at org.apache.hadoop.hive.ql.session.SessionState.init(SessionState.java:202) at org.apache.hadoop.hive.cli.CliSessionState.init(CliSessionState.java:86) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:635) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Backing the patch out makes everything work again. get_json_object and json_tuple return null in the presence of new line characters - Key: HIVE-3581 URL: https://issues.apache.org/jira/browse/HIVE-3581 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3581.1.patch.txt This was introduced when these functions were updated to use Jackson. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3591) set hive.security.authorization.enabled can be executed by any user
Dev Gupta created HIVE-3591: --- Summary: set hive.security.authorization.enabled can be executed by any user Key: HIVE-3591 URL: https://issues.apache.org/jira/browse/HIVE-3591 Project: Hive Issue Type: Bug Components: CLI, Clients, JDBC Affects Versions: 0.7.1 Environment: RHEL 5.6 CDH U3 Reporter: Dev Gupta The property hive.security.authorization.enabled can be set to true or false, by any user on the CLI, thus circumventing any previously set grants and authorizations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #170
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/170/ -- [...truncated 10215 lines...] [echo] Project: odbc [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/odbc/src/conf does not exist. ivy-resolve-test: [echo] Project: odbc ivy-retrieve-test: [echo] Project: odbc compile-test: [echo] Project: odbc create-dirs: [echo] Project: serde [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/serde/src/test/resources does not exist. init: [echo] Project: serde ivy-init-settings: [echo] Project: serde ivy-resolve: [echo] Project: serde [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml [ivy:report] Processing /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-serde-default.xml to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-serde-default.html ivy-retrieve: [echo] Project: serde dynamic-serde: compile: [echo] Project: serde ivy-resolve-test: [echo] Project: serde ivy-retrieve-test: [echo] Project: serde compile-test: [echo] Project: serde [javac] Compiling 26 source files to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/serde/test/classes [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. create-dirs: [echo] Project: service [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/service/src/test/resources does not exist. init: [echo] Project: service ivy-init-settings: [echo] Project: service ivy-resolve: [echo] Project: service [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml [ivy:report] Processing /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-service-default.html ivy-retrieve: [echo] Project: service compile: [echo] Project: service ivy-resolve-test: [echo] Project: service ivy-retrieve-test: [echo] Project: service compile-test: [echo] Project: service [javac] Compiling 2 source files to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/service/test/classes test: [echo] Project: hive test-shims: [echo] Project: hive test-conditions: [echo] Project: shims gen-test: [echo] Project: shims create-dirs: [echo] Project: shims [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/test/resources does not exist. init: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-resolve: [echo] Project: shims [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml [ivy:report] Processing /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-shims-default.html ivy-retrieve: [echo] Project: shims compile: [echo] Project: shims [echo] Building shims 0.20 build_shims: [echo] Project: shims [echo] Compiling /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java against hadoop 0.20.2 (/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-0.20.2) ivy-init-settings: [echo] Project: shims ivy-resolve-hadoop-shim: [echo] Project: shims [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml ivy-retrieve-hadoop-shim: [echo] Project: shims [echo] Building shims 0.20S build_shims: [echo] Project: shims [echo] Compiling
[jira] [Updated] (HIVE-3276) optimize union sub-queries
[ https://issues.apache.org/jira/browse/HIVE-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3276: - Status: Patch Available (was: Open) addressed comments optimize union sub-queries -- Key: HIVE-3276 URL: https://issues.apache.org/jira/browse/HIVE-3276 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3276.10.patch, hive.3276.11.patch, hive.3276.12.patch, hive.3276.13.patch, HIVE-3276.1.patch, hive.3276.2.patch, hive.3276.3.patch, hive.3276.4.patch, hive.3276.5.patch, hive.3276.6.patch, hive.3276.7.patch, hive.3276.8.patch, hive.3276.9.patch It might be a good idea to optimize simple union queries containing map-reduce jobs in at least one of the sub-qeuries. For eg: a query like: insert overwrite table T1 partition P1 select * from ( subq1 union all subq2 ) u; today creates 3 map-reduce jobs, one for subq1, another for subq2 and the final one for the union. It might be a good idea to optimize this. Instead of creating the union task, it might be simpler to create a move task (or something like a move task), where the outputs of the two sub-queries will be moved to the final directory. This can easily extend to more than 2 sub-queries in the union. This is very useful if there is a select * followed by filesink after the union. This can be independently useful, and also be used to optimize the skewed joins https://cwiki.apache.org/Hive/skewed-join-optimization.html. If there is a select, filter between the union and the filesink, the select and the filter can be moved before the union, and the follow-up job can still be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3591) set hive.security.authorization.enabled can be executed by any user
[ https://issues.apache.org/jira/browse/HIVE-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shreepadma Venugopalan updated HIVE-3591: - Component/s: Authorization set hive.security.authorization.enabled can be executed by any user --- Key: HIVE-3591 URL: https://issues.apache.org/jira/browse/HIVE-3591 Project: Hive Issue Type: Bug Components: Authorization, CLI, Clients, JDBC Affects Versions: 0.7.1 Environment: RHEL 5.6 CDH U3 Reporter: Dev Gupta Labels: Authorization, Security The property hive.security.authorization.enabled can be set to true or false, by any user on the CLI, thus circumventing any previously set grants and authorizations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3563: -- Attachment: HIVE-3563-2.patch Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-1.patch, HIVE-3563-2.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478218#comment-13478218 ] Prasad Mujumdar commented on HIVE-3563: --- Review request on https://reviews.apache.org/r/7517/ Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-1.patch, HIVE-3563-2.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3563: -- Status: Patch Available (was: Open) Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-1.patch, HIVE-3563-2.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3551) Add more tests where output of sort merge join is sorted
[ https://issues.apache.org/jira/browse/HIVE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3551: - Attachment: hive.3551.2.patch Add more tests where output of sort merge join is sorted Key: HIVE-3551 URL: https://issues.apache.org/jira/browse/HIVE-3551 Project: Hive Issue Type: Test Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3551.1.patch, hive.3551.2.patch As a follow-up to HIVE-3536, it would be good to have tests where a join is being performed between a single partition of the big table with multiple partitions of the small tables. The output should still be sorted -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3551) Add more tests where output of sort merge join is sorted
[ https://issues.apache.org/jira/browse/HIVE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3551: - Attachment: hive.3551.3.patch Add more tests where output of sort merge join is sorted Key: HIVE-3551 URL: https://issues.apache.org/jira/browse/HIVE-3551 Project: Hive Issue Type: Test Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3551.1.patch, hive.3551.2.patch, hive.3551.3.patch As a follow-up to HIVE-3536, it would be good to have tests where a join is being performed between a single partition of the big table with multiple partitions of the small tables. The output should still be sorted -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3551) Add more tests where output of sort merge join is sorted
[ https://issues.apache.org/jira/browse/HIVE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3551: - Status: Patch Available (was: Open) addressed comments Add more tests where output of sort merge join is sorted Key: HIVE-3551 URL: https://issues.apache.org/jira/browse/HIVE-3551 Project: Hive Issue Type: Test Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3551.1.patch, hive.3551.2.patch, hive.3551.3.patch As a follow-up to HIVE-3536, it would be good to have tests where a join is being performed between a single partition of the big table with multiple partitions of the small tables. The output should still be sorted -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3590: TCP KeepAlive and connection timeout for the HiveServer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7634/ --- (Updated Oct. 17, 2012, 8:14 p.m.) Review request for hive. Changes --- fixed typo. Description --- Simple implementation of TCP KeepAlive and socket timeout for the HiveServer in order to avoid half open connections or connection drops during long running queries that use HiveServer. Two new settings have been added in order to control this functionality: hive.server.read.socket.timeout and hive.server.tcp.keepalive the proposal is to have hive.server.tcp.keepalive set by default to true. Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f86d6a7 conf/hive-default.xml.template 4a59fb6 service/src/java/org/apache/hadoop/hive/service/HiveServer.java 2b2f681 Diff: https://reviews.apache.org/r/7634/diff/ Testing --- Thanks, Esteban Gutierrez
[jira] [Commented] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478343#comment-13478343 ] Kevin Wilfong commented on HIVE-3544: - https://reviews.facebook.net/D6063 union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3581.1.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3544: Attachment: HIVE-3581.1.patch.txt union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3581.1.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3544: Status: Patch Available (was: Open) union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3581.1.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478346#comment-13478346 ] Kevin Wilfong commented on HIVE-3544: - I tracked the problem down to two causes. 1) In the genUnionPlan, when preparing the ColumnInfo objects to be used to generate the RowResolver for the Union operator, it actually changes the ColumnInfo objects of the left operator's RowResolver to have the common class as its type. This would cause it to get serialized wrong in the intermediate FileSink operator between map reduce jobs (as was the case when the left subquery of the union involved a join). 2) The common class for a column of the Union operator would get determined once at compile time and again later at run time using different functions which could return different classes (for instance when the type on one side was a double and on the other it was a string). This caused the union operator to return objects with a different type from what the RowResolver specified causing serialization errors/failures. To fix 1) I added the ability to clone a ColumnInfo, and in the SemanticAnalyzer the left operator's ColumnInfo objects are now cloned before being modified. To fix 2) I added Select operators between the input operators and the union operator. These select operators cast the input columns to the types determined at compile time if they do not match, otherwise they simply forward the value. Now the conversion in the union operator is only needed to alter the the type of the ObjectInspector, not the type of the column. union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3581.1.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3276) optimize union sub-queries
[ https://issues.apache.org/jira/browse/HIVE-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478349#comment-13478349 ] Kevin Wilfong commented on HIVE-3276: - I was referring to HIVE-3544 in the above comment. optimize union sub-queries -- Key: HIVE-3276 URL: https://issues.apache.org/jira/browse/HIVE-3276 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3276.10.patch, hive.3276.11.patch, hive.3276.12.patch, hive.3276.13.patch, HIVE-3276.1.patch, hive.3276.2.patch, hive.3276.3.patch, hive.3276.4.patch, hive.3276.5.patch, hive.3276.6.patch, hive.3276.7.patch, hive.3276.8.patch, hive.3276.9.patch It might be a good idea to optimize simple union queries containing map-reduce jobs in at least one of the sub-qeuries. For eg: a query like: insert overwrite table T1 partition P1 select * from ( subq1 union all subq2 ) u; today creates 3 map-reduce jobs, one for subq1, another for subq2 and the final one for the union. It might be a good idea to optimize this. Instead of creating the union task, it might be simpler to create a move task (or something like a move task), where the outputs of the two sub-queries will be moved to the final directory. This can easily extend to more than 2 sub-queries in the union. This is very useful if there is a select * followed by filesink after the union. This can be independently useful, and also be used to optimize the skewed joins https://cwiki.apache.org/Hive/skewed-join-optimization.html. If there is a select, filter between the union and the filesink, the select and the filter can be moved before the union, and the follow-up job can still be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3592) where clause is ignored when limit is used and query runs locally
Alina created HIVE-3592: --- Summary: where clause is ignored when limit is used and query runs locally Key: HIVE-3592 URL: https://issues.apache.org/jira/browse/HIVE-3592 Project: Hive Issue Type: Bug Components: Query Processor, SQL Environment: AWS Reporter: Alina select * from tbl where blah='foo' limit 4; results in displaying first 4 rows of the tbl that runs locally, the blah='foo' condition is ignored if I change the select to specify columns which will essentially force it to not run locally but do map reduce works fine select column1 as test, * from tbl where blah='foo' limit 4; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3570) Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr
[ https://issues.apache.org/jira/browse/HIVE-3570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3570: -- Attachment: HIVE-3570.D5985.2.patch satadru updated the revision HIVE-3570 [jira] Hive changes for Optr level stats. Reviewers: njain Update with CR comments by Namit REVISION DETAIL https://reviews.facebook.net/D5985 AFFECTED FILES ql/src/test/results/clientpositive/OptrStatGroupBy.q.out ql/src/test/org/apache/hadoop/hive/ql/hooks/OptrStatGroupByHook.java ql/src/test/queries/clientpositive/OptrStatGroupBy.q ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java To: njain, satadru Cc: JIRA Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr --- Key: HIVE-3570 URL: https://issues.apache.org/jira/browse/HIVE-3570 Project: Hive Issue Type: Improvement Components: Statistics Affects Versions: 0.9.0 Reporter: Satadru Pan Assignee: Satadru Pan Priority: Minor Attachments: HIVE-3570.1.patch.txt, HIVE-3570.D5985.1.patch, HIVE-3570.D5985.2.patch Requirement: Collect Operator specific stats for hive queries. Use the counter framework available in Hive Operator.java to accomplish that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3570) Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr
[ https://issues.apache.org/jira/browse/HIVE-3570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478387#comment-13478387 ] Phabricator commented on HIVE-3570: --- satadru has commented on the revision HIVE-3570 [jira] Hive changes for Optr level stats. @njain : I have done the changes and added an hook. Whenever I am executing the hook from outside(ie hive cli) everything works fine. But when it is run from the ant test -Dtestcase=... command, the Counter object in the Operator.java coming as null. (Probable reason: the unit test is not executing the query in actual hadoop cluster, without which counters are always null). Please suggest if there is any way to get around this problem. I tried searching for unit test involving counters in Operator.java, but couldn't find. REVISION DETAIL https://reviews.facebook.net/D5985 To: njain, satadru Cc: JIRA Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr --- Key: HIVE-3570 URL: https://issues.apache.org/jira/browse/HIVE-3570 Project: Hive Issue Type: Improvement Components: Statistics Affects Versions: 0.9.0 Reporter: Satadru Pan Assignee: Satadru Pan Priority: Minor Attachments: HIVE-3570.1.patch.txt, HIVE-3570.D5985.1.patch, HIVE-3570.D5985.2.patch Requirement: Collect Operator specific stats for hive queries. Use the counter framework available in Hive Operator.java to accomplish that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3593) Output files of SMB join grow indefinitely
Kevin Wilfong created HIVE-3593: --- Summary: Output files of SMB join grow indefinitely Key: HIVE-3593 URL: https://issues.apache.org/jira/browse/HIVE-3593 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong The output files of a SMB join are prefixed by the big table's partition spec that was used to create them. The length of the bucket number portion of the file name is updated to be the same length as the length of the task ID. Since the task ID is the name of the file, this means that if the output of a SMB join is used as the big table of another SMB join, the output files will increase by the size of the original partition spec. Compound this and the file size can grow indefinitely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3525) Avro Maps with Nullable Values fail with NPE
[ https://issues.apache.org/jira/browse/HIVE-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478416#comment-13478416 ] Carl Steinbach commented on HIVE-3525: -- @Jakob: thanks for reviewing this! @Sean: +1. Will commit if tests pass. Avro Maps with Nullable Values fail with NPE Key: HIVE-3525 URL: https://issues.apache.org/jira/browse/HIVE-3525 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Sean Busbey Attachments: HIVE-3525.1.patch.txt, HIVE-3525.2.patch.txt When working against current trunk@1393794, using a backing Avro schema that has a Map field with nullable values causes a NPE on deserialization when the map contains a null value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3593) Output files of SMB join grow indefinitely
[ https://issues.apache.org/jira/browse/HIVE-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478419#comment-13478419 ] Kevin Wilfong commented on HIVE-3593: - Mildly related, if only one partition of the big table is used as input to the SMB join, there is no need to prefix the file name with the partition spec. Output files of SMB join grow indefinitely -- Key: HIVE-3593 URL: https://issues.apache.org/jira/browse/HIVE-3593 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong The output files of a SMB join are prefixed by the big table's partition spec that was used to create them. The length of the bucket number portion of the file name is updated to be the same length as the length of the task ID. Since the task ID is the name of the file, this means that if the output of a SMB join is used as the big table of another SMB join, the output files will increase by the size of the original partition spec. Compound this and the file size can grow indefinitely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1743 - Still Failing
Changes for Build #1739 Changes for Build #1740 [namit] HIVE-3581 get_json_object and json_tuple return null in the presence of new line characters (Kevin Wilfong via namit) Changes for Build #1741 Changes for Build #1742 [kevinwilfong] HIVE-967. Implement show create table (Feng Lu via kevinwilfong) Changes for Build #1743 2 tests failed. REGRESSION: org.apache.hadoop.hive.metastore.TestMetaStoreEventListener.testListener Error Message: java.net.SocketException: Broken pipe Stack Trace: org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147) at org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163) at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:186) at org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:92) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_get_database(ThriftHiveMetastore.java:372) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:364) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:705) at org.apache.hadoop.hive.metastore.TestMetaStoreEventListener.testListener(TestMetaStoreEventListener.java:190) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) Caused by: java.net.SocketException: Broken pipe at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) at java.net.SocketOutputStream.write(SocketOutputStream.java:136) at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145) ... 23 more REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1 Error Message: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. Stack Trace: junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259) at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268) at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324) at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1743) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1743/ to view the results.
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #170
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/170/ -- [...truncated 36586 lines...] [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/jenkins/hive_2012-10-17_15-23-36_921_6479299663387946634/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/170/artifact/hive/build/service/tmp/hive_job_log_jenkins_201210171523_2134340429.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] Copying file: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] PREHOOK: Output: default@testhivedrivertable [junit] Copying data from https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/jenkins/hive_2012-10-17_15-23-41_197_1188942045386653891/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/jenkins/hive_2012-10-17_15-23-41_197_1188942045386653891/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/170/artifact/hive/build/service/tmp/hive_job_log_jenkins_201210171523_1163475288.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/170/artifact/hive/build/service/tmp/hive_job_log_jenkins_201210171523_1136570347.txt [junit] Copying file: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/170/artifact/hive/build/service/tmp/hive_job_log_jenkins_201210171523_1737243355.txt [junit] PREHOOK: query: drop table testhivedrivertable
[jira] [Resolved] (HIVE-3593) Output files of SMB join grow indefinitely
[ https://issues.apache.org/jira/browse/HIVE-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong resolved HIVE-3593. - Resolution: Not A Problem Actually, the regex to get the task ID already avoids this problem. Output files of SMB join grow indefinitely -- Key: HIVE-3593 URL: https://issues.apache.org/jira/browse/HIVE-3593 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong The output files of a SMB join are prefixed by the big table's partition spec that was used to create them. The length of the bucket number portion of the file name is updated to be the same length as the length of the task ID. Since the task ID is the name of the file, this means that if the output of a SMB join is used as the big table of another SMB join, the output files will increase by the size of the original partition spec. Compound this and the file size can grow indefinitely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3582) NPE in union processing followed by lateral view followed by 2 group bys
[ https://issues.apache.org/jira/browse/HIVE-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-3582: Status: Patch Available (was: Open) NPE in union processing followed by lateral view followed by 2 group bys Key: HIVE-3582 URL: https://issues.apache.org/jira/browse/HIVE-3582 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Navis Attachments: HIVE-3582.D6051.1.patch EXPLAIN SELECT e.key, e.arr_ele, count(1) FROM ( SELECT d.key as key, d.arr_ele as arr_ele, d.value as value, count(1) as cnt FROM ( SELECT c.arr_ele as arr_ele, a.key as key, a.value as value FROM ( SELECT key, value, array(1,2,3) as arr FROM src UNION ALL SELECT key, value, array(1,2,3) as arr FROM srcpart WHERE ds = '2008-04-08' and hr='12' ) a LATERAL VIEW EXPLODE(arr) c AS arr_ele ) d group by d.key, d.arr_ele, d.value ) e group by e.key, e.arr_ele; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3570) Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr
[ https://issues.apache.org/jira/browse/HIVE-3570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satadru Pan updated HIVE-3570: -- Status: Patch Available (was: Open) Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr --- Key: HIVE-3570 URL: https://issues.apache.org/jira/browse/HIVE-3570 Project: Hive Issue Type: Improvement Components: Statistics Affects Versions: 0.9.0 Reporter: Satadru Pan Assignee: Satadru Pan Priority: Minor Attachments: HIVE-3570.1.patch.txt, HIVE-3570.D5985.1.patch, HIVE-3570.D5985.2.patch Requirement: Collect Operator specific stats for hive queries. Use the counter framework available in Hive Operator.java to accomplish that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3584) Merging compare predicates
[ https://issues.apache.org/jira/browse/HIVE-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3584: -- Attachment: HIVE-3584.D6069.1.patch navis requested code review of HIVE-3584 [jira] Merging compare predicates. Reviewers: JIRA DPAL-1914 Merging compare predicates Some compare predicates, for example 'a10 AND a20' can be merged into 'a20' TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D6069 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrOpProcFactory.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverCommonJoin.java ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeConstantDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java ql/src/test/org/apache/hadoop/hive/ql/plan/TestExprNodeDescUtils.java ql/src/test/results/clientpositive/auto_join12.q.out ql/src/test/results/clientpositive/auto_join16.q.out ql/src/test/results/clientpositive/auto_join20.q.out ql/src/test/results/clientpositive/auto_join28.q.out ql/src/test/results/clientpositive/auto_join29.q.out ql/src/test/results/clientpositive/index_auto_mult_tables.q.out ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out ql/src/test/results/clientpositive/join12.q.out ql/src/test/results/clientpositive/join16.q.out ql/src/test/results/clientpositive/join20.q.out ql/src/test/results/clientpositive/join40.q.out ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out ql/src/test/results/clientpositive/ppd_gby.q.out ql/src/test/results/clientpositive/ppd_gby2.q.out ql/src/test/results/clientpositive/ppd_gby_join.q.out ql/src/test/results/clientpositive/ppd_join.q.out ql/src/test/results/clientpositive/ppd_udf_col.q.out ql/src/test/results/clientpositive/ppd_union.q.out ql/src/test/results/clientpositive/sample8.q.out ql/src/test/results/clientpositive/skewjoin.q.out ql/src/test/results/clientpositive/udf_count.q.out ql/src/test/results/clientpositive/udf_percentile.q.out serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/14511/ To: JIRA, navis Merging compare predicates -- Key: HIVE-3584 URL: https://issues.apache.org/jira/browse/HIVE-3584 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3584.D6069.1.patch Some compare predicates, for example 'a10 AND a20' can be merged into 'a20' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3584) Merging compare predicates
[ https://issues.apache.org/jira/browse/HIVE-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-3584: Status: Patch Available (was: Open) This patch contains input pruning for empty table, which means predicate for TS is FALSE, it does not read anything from it. This was supported behavior for partitioned table but for non-partitioned empty table, it was not. Merging compare predicates -- Key: HIVE-3584 URL: https://issues.apache.org/jira/browse/HIVE-3584 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3584.D6069.1.patch Some compare predicates, for example 'a10 AND a20' can be merged into 'a20' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2540) LATERAL VIEW with EXPLODE produces ConcurrentModificationException
[
https://issues.apache.org/jira/browse/HIVE-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478509#comment-13478509
]
Navis commented on HIVE-2540:
-
It's not on hive-0.9.x. Would be included hive-0.10.0 maybe?
LATERAL VIEW with EXPLODE produces ConcurrentModificationException
--
Key: HIVE-2540
URL: https://issues.apache.org/jira/browse/HIVE-2540
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.7.1, 0.9.0
Reporter: David Phillips
Assignee: Navis
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2540.D2805.1.patch,
ASF.LICENSE.NOT.GRANTED--HIVE-2540.D2805.2.patch,
ASF.LICENSE.NOT.GRANTED--HIVE-2540.D2805.3.patch
The following produces {{ConcurrentModificationException}} on the {{for}}
loop inside EXPLODE:
{code}
create table foo as select array(1, 2) a from src limit 1;
select a, x.b from foo lateral view explode(a) x as b;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3544: Attachment: HIVE-3544.2.patch.txt union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3544.1.patch.txt, HIVE-3544.2.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3544: Attachment: (was: HIVE-3581.1.patch.txt) union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3544.1.patch.txt, HIVE-3544.2.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3544: Attachment: HIVE-3544.1.patch.txt union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3544.1.patch.txt, HIVE-3544.2.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3544) union involving double column with a map join subquery will fail or give wrong results
[ https://issues.apache.org/jira/browse/HIVE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478511#comment-13478511 ] Kevin Wilfong commented on HIVE-3544: - Updated my checkout, and uploaded a new patch. union involving double column with a map join subquery will fail or give wrong results -- Key: HIVE-3544 URL: https://issues.apache.org/jira/browse/HIVE-3544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3544.1.patch.txt, HIVE-3544.2.patch.txt The following query fails: select * from (select cast(a.key as bigint) as key from src a join src b on a.key = b.key union all select cast(key as double) as key from src)a The following query gives wrong results: select * from (select cast(a.key as bigint) as key, cast(b.key as double) as value from src a join src b on a.key = b.key union all select cast(key as double) as key, cast(key as string) as value from src)a But the following query runs fine: select * from (select cast(a.key as bigint) as key from src a union all select cast(key as double) as key from src)a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3551) Add more tests where output of sort merge join is sorted
[ https://issues.apache.org/jira/browse/HIVE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478540#comment-13478540 ] Kevin Wilfong commented on HIVE-3551: - +1 Add more tests where output of sort merge join is sorted Key: HIVE-3551 URL: https://issues.apache.org/jira/browse/HIVE-3551 Project: Hive Issue Type: Test Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3551.1.patch, hive.3551.2.patch, hive.3551.3.patch As a follow-up to HIVE-3536, it would be good to have tests where a join is being performed between a single partition of the big table with multiple partitions of the small tables. The output should still be sorted -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3563: - Status: Open (was: Patch Available) @Prasad: I left some more comments on reviewboard. Thanks. Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-1.patch, HIVE-3563-2.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3535) derby metastore upgrade script throw errors when updating from 0.7 to 0.8
[ https://issues.apache.org/jira/browse/HIVE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478559#comment-13478559 ] Carl Steinbach commented on HIVE-3535: -- +1. Will commit if tests pass. Thanks. derby metastore upgrade script throw errors when updating from 0.7 to 0.8 - Key: HIVE-3535 URL: https://issues.apache.org/jira/browse/HIVE-3535 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.9.0 Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Fix For: 0.10.0 Attachments: HIVE-3535.1.patch.txt $DERBY_HOME/bin/ij upgrade.sql ij version 10.4 ij CONNECT 'jdbc:derby:/var/lib/hive/metastore/metastore_db'; ij RUN '/usr/lib/hive/scripts/metastore/upgrade/derby/upgrade-0.7.0-to-0.8.0.derby.sql'; ij -- Upgrade MetaStore schema from 0.7.0 to 0.8.0 RUN '008-HIVE-2246.derby.sql'; ij /* * Creates the following tables: * - CDS * - COLUMNS_V2 * The new columns table is called COLUMNS_V2 * because many columns are removed, and the schema is changed. * It'd take too long to migrate and keep the same table. */ CREATE TABLE CDS ( CD_ID bigint NOT NULL, PRIMARY KEY (CD_ID) ); 0 rows inserted/updated/deleted ij CREATE TABLE COLUMNS_V2 ( CD_ID bigint NOT NULL, COMMENT varchar(4000), COLUMN_NAME varchar(128) NOT NULL, TYPE_NAME varchar(4000), INTEGER_IDX INTEGER NOT NULL, PRIMARY KEY (CD_ID, COLUMN_NAME) ); 0 rows inserted/updated/deleted ij ALTER TABLE COLUMNS_V2 ADD CONSTRAINT COLUMNS_V2_FK1 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID) ON DELETE NO ACTION ON UPDATE NO ACTION ; 0 rows inserted/updated/deleted ij /* Alter the SDS table to: * - add the column CD_ID * - add a foreign key on CD_ID * - create an index on CD_ID */ ALTER TABLE SDS ADD COLUMN CD_ID bigint ; 0 rows inserted/updated/deleted ij ALTER TABLE SDS ADD CONSTRAINT SDS_FK2 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID) ; 0 rows inserted/updated/deleted ij /* * Migrate the TBLS table * Add entries into CDS. * Populate the CD_ID field in SDS for tables * Add entires to COLUMNS_V2 based on this table's sd's columns */ /* In the migration, there is a 1:1 mapping between CD_ID and SD_ID * for tables. For speed, just let CD_ID = SD_ID for tables */ INSERT INTO CDS (CD_ID) SELECT t.SD_ID FROM TBLS t WHERE t.SD_ID IS NOT NULL ORDER BY t.SD_ID; ERROR 42X01: Syntax error: Encountered ORDER at line 13, column 54. ij UPDATE SDS SET CD_ID = SD_ID WHERE SD_ID in (SELECT t.SD_ID FROM TBLS t WHERE t.SD_ID IS NOT NULL ORDER BY t.SD_ID); ERROR 42X01: Syntax error: Encountered ORDER at line 4, column 55. ij INSERT INTO COLUMNS_V2 (CD_ID, COMMENT, COLUMN_NAME, TYPE_NAME, INTEGER_IDX) SELECT c.SD_ID, c.COMMENT, c.COLUMN_NAME, c.TYPE_NAME, c.INTEGER_IDX FROM COLUMNS c JOIN TBLS t ON t.SD_ID = c.SD_ID ; ERROR 23503: INSERT on table 'COLUMNS_V2' caused a violation of foreign key constraint 'COLUMNS_V2_FK1' for key (1). The statement has been rolled back. ij /* * Migrate the partitions. * Update the partitions' SDS to use the parent tables' CD_ID BEGIN * Derby does not allow joins in update statements, * so we have to make a temporary tableh */ DECLARE GLOBAL TEMPORARY TABLE TMP_TBL ( SD_ID bigint not null, CD_ID bigint not null ) ON COMMIT PRESERVE ROWS NOT LOGGED; 0 rows inserted/updated/deleted ij INSERT INTO SESSION.TMP_TBL SELECT p.SD_ID, sds.CD_ID FROM PARTITIONS p JOIN TBLS t ON t.TBL_ID = p.TBL_ID JOIN SDS sds on t.SD_ID = sds.SD_ID WHERE p.SD_ID IS NOT NULL; ERROR 23502: Column 'CD_ID' cannot accept a NULL value. ij UPDATE SDS sd SET sd.CD_ID = (SELECT tt.CD_ID FROM SESSION.TMP_TBL tt WHERE tt.SD_ID = sd.SD_ID) WHERE sd.SD_ID IN (SELECT SD_ID FROM SESSION.TMP_TBL); 0 rows inserted/updated/deleted WARNING 02000: No row was found for FETCH, UPDATE or DELETE; or the result of a query is an empty table. ij /* * Migrate IDXS */ INSERT INTO CDS (CD_ID) SELECT i.SD_ID FROM IDXS i WHERE i.SD_ID IS NOT NULL ORDER BY i.SD_ID; ERROR 42X01: Syntax error: Encountered ORDER at line 5, column 54. ij UPDATE SDS SET CD_ID = SD_ID WHERE SD_ID in (SELECT i.SD_ID FROM IDXS i WHERE i.SD_ID IS NOT NULL ORDER BY i.SD_ID); ERROR 42X01: Syntax error: Encountered ORDER at line 4, column 55. ij INSERT INTO COLUMNS_V2 (CD_ID, COMMENT, COLUMN_NAME, TYPE_NAME, INTEGER_IDX) SELECT c.SD_ID, c.COMMENT, c.COLUMN_NAME, c.TYPE_NAME, c.INTEGER_IDX FROM COLUMNS c JOIN IDXS i ON i.SD_ID = c.SD_ID ; ERROR 42X05: Table/View 'IDXS' does not exist. ij /* * rename the old COLUMNS table */ RENAME TABLE COLUMNS TO COLUMNS_OLD; 0 rows inserted/updated/deleted ij
[jira] [Created] (HIVE-3594) Hive
Daisy.Yuan created HIVE-3594: Summary: Hive Key: HIVE-3594 URL: https://issues.apache.org/jira/browse/HIVE-3594 Project: Hive Issue Type: Bug Components: CLI, Clients, Configuration, Indexing, JDBC, Locking, Metastore, Query Processor, Serializers/Deserializers, Server Infrastructure, Shims, SQL, Statistics, StorageHandler, UDF Affects Versions: 0.9.0, 0.10.0, 0.9.1 Reporter: Daisy.Yuan Fix For: 0.10.0, 0.9.1, 0.9.0, 0.8.1, 0.8.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2715) Upgrade Thrift dependency to 0.9.0
[ https://issues.apache.org/jira/browse/HIVE-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478584#comment-13478584 ] Carl Steinbach commented on HIVE-2715: -- Thrift 0.9.0 has been released. We should Hive's Thrift dependency. Upgrade Thrift dependency to 0.9.0 -- Key: HIVE-2715 URL: https://issues.apache.org/jira/browse/HIVE-2715 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.8.0 Reporter: Mithun Radhakrishnan I work on HCatalog (0.2). Recently, we ran into HCat_server running out of memory every few days, and it boiled down to a bug in thrift, (THRIFT-1468, recently fixed). HCat-0.2-branch depends on Hive-0.8, which in turn depends on thrift-0.5.0. (The bug also exists on 0.7.0.) May I please enquire if Hive can't depend on a more current version of thrift? (Does it break the metastore?) I'm afraid I'm not privy to the reasoning behind Hive's dependency on a slightly dated thrift-lib. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-2715) Upgrade Thrift dependency to 0.9.0
[ https://issues.apache.org/jira/browse/HIVE-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478584#comment-13478584 ] Carl Steinbach edited comment on HIVE-2715 at 10/18/12 1:46 AM: Thrift 0.9.0 has been released. We should update Hive's Thrift dependency to 0.9.0. was (Author: cwsteinbach): Thrift 0.9.0 has been released. We should Hive's Thrift dependency. Upgrade Thrift dependency to 0.9.0 -- Key: HIVE-2715 URL: https://issues.apache.org/jira/browse/HIVE-2715 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.8.0 Reporter: Mithun Radhakrishnan I work on HCatalog (0.2). Recently, we ran into HCat_server running out of memory every few days, and it boiled down to a bug in thrift, (THRIFT-1468, recently fixed). HCat-0.2-branch depends on Hive-0.8, which in turn depends on thrift-0.5.0. (The bug also exists on 0.7.0.) May I please enquire if Hive can't depend on a more current version of thrift? (Does it break the metastore?) I'm afraid I'm not privy to the reasoning behind Hive's dependency on a slightly dated thrift-lib. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3594) Group by Timestamp or STRING which format contains HH:MM:SS occurs URISyntaxException
[
https://issues.apache.org/jira/browse/HIVE-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daisy.Yuan updated HIVE-3594:
-
Component/s: (was: Locking)
(was: StorageHandler)
(was: Shims)
(was: SQL)
(was: Statistics)
(was: JDBC)
(was: Indexing)
(was: CLI)
(was: UDF)
(was: Configuration)
(was: Server Infrastructure)
(was: Clients)
(was: Serializers/Deserializers)
(was: Metastore)
Description:
create table test (no int, name string) partitioned by (pts string) row format
delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts2.txt' into table test
partition(pts='21:25:12');
load data local inpath '/opt/files/groupbyts3.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts4.txt' into table test
partition(pts='21:25:12');
when I execute “select * from test group by pts;”, it will occur as follows
exception.
at org.apache.hadoop.fs.Path.initialize(Path.java:157)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.exec.Utilities.getInputSummary(Utilities.java:1667)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.estimateNumberOfReducers(MapRedTask.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.setNumberOfReducers(MapRedTask.java:400)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:93)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1329)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:954)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
fake-path-metadata-only-query-default.test{pts=12:11:30%7D
at java.net.URI.checkPath(URI.java:1788)
at java.net.URI.init(URI.java:734)
at org.apache.hadoop.fs.Path.initialize(Path.java:154)
... 19 more
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
When PhysicalOptimizer optimizes GroupByOperator, according to default
parameters hive.optimize.metadataonly = true, MetadataOnlyOptimizer will be
enabled. The MetadataOnlyOptimizer will change the partition alias desc. The
partition alies
hdfs://160.165.0.1:9000/user/hive/warehouse/test/pts=12%3A11%3A30 is changed
into
fake-path-metadata-only-query-default.test{pts=12:11:30}. When construct uri
through new partition alies, it must occur java.net.URISyntaxException.
Affects Version/s: (was: 0.9.1)
(was: 0.10.0)
Fix Version/s: (was: 0.9.1)
(was: 0.10.0)
(was: 0.8.1)
(was: 0.9.0)
(was: 0.8.0)
Summary: Group by Timestamp or STRING which format contains
HH:MM:SS occurs URISyntaxException (was: Hive)
Group by Timestamp or STRING which format contains HH:MM:SS occurs
URISyntaxException
---
Key: HIVE-3594
URL: https://issues.apache.org/jira/browse/HIVE-3594
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Reporter: Daisy.Yuan
create table test (no int, name string) partitioned by (pts string) row
format delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
[jira] [Updated] (HIVE-3589) describe/show partition/show tblproperties command should accept database name
[ https://issues.apache.org/jira/browse/HIVE-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-3589: Summary: describe/show partition/show tblproperties command should accept database name (was: describe command not giving the details when called as describe dbname.tablename) describe/show partition/show tblproperties command should accept database name -- Key: HIVE-3589 URL: https://issues.apache.org/jira/browse/HIVE-3589 Project: Hive Issue Type: Bug Components: Metastore, Query Processor Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Priority: Minor describe command not giving the details when called as describe dbname.tablename. Throwing the error Table dbname not found. Ex: hive -e describe masterdb.table1 will throw error Table masterdb not found -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3589) describe/show partition/show tblproperties command should accept database name
[ https://issues.apache.org/jira/browse/HIVE-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis reassigned HIVE-3589: --- Assignee: Navis describe/show partition/show tblproperties command should accept database name -- Key: HIVE-3589 URL: https://issues.apache.org/jira/browse/HIVE-3589 Project: Hive Issue Type: Bug Components: Metastore, Query Processor Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Assignee: Navis Priority: Minor describe command not giving the details when called as describe dbname.tablename. Throwing the error Table dbname not found. Ex: hive -e describe masterdb.table1 will throw error Table masterdb not found -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3594) When Group by Partition column type is Timestamp or STRING which format contains HH:MM:SS, It will occur URISyntaxException
[
https://issues.apache.org/jira/browse/HIVE-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daisy.Yuan updated HIVE-3594:
-
Summary: When Group by Partition column type is Timestamp or STRING which
format contains HH:MM:SS, It will occur URISyntaxException (was: Group by
Timestamp or STRING which format contains HH:MM:SS occurs URISyntaxException)
When Group by Partition column type is Timestamp or STRING which format
contains HH:MM:SS, It will occur URISyntaxException
-
Key: HIVE-3594
URL: https://issues.apache.org/jira/browse/HIVE-3594
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Reporter: Daisy.Yuan
create table test (no int, name string) partitioned by (pts string) row
format delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts2.txt' into table test
partition(pts='21:25:12');
load data local inpath '/opt/files/groupbyts3.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts4.txt' into table test
partition(pts='21:25:12');
when I execute “select * from test group by pts;”, it will occur as follows
exception.
at org.apache.hadoop.fs.Path.initialize(Path.java:157)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.exec.Utilities.getInputSummary(Utilities.java:1667)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.estimateNumberOfReducers(MapRedTask.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.setNumberOfReducers(MapRedTask.java:400)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:93)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1329)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:954)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
fake-path-metadata-only-query-default.test{pts=12:11:30%7D
at java.net.URI.checkPath(URI.java:1788)
at java.net.URI.init(URI.java:734)
at org.apache.hadoop.fs.Path.initialize(Path.java:154)
... 19 more
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
When PhysicalOptimizer optimizes GroupByOperator, according to default
parameters hive.optimize.metadataonly = true, MetadataOnlyOptimizer will be
enabled. The MetadataOnlyOptimizer will change the partition alias desc. The
partition alies
hdfs://160.165.0.1:9000/user/hive/warehouse/test/pts=12%3A11%3A30 is
changed into
fake-path-metadata-only-query-default.test{pts=12:11:30}. When construct uri
through new partition alies, it must occur java.net.URISyntaxException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3594) When Group by Partition Column Type is Timestamp or STRING Which Format contains HH:MM:SS, It will occur URISyntaxException
[
https://issues.apache.org/jira/browse/HIVE-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daisy.Yuan updated HIVE-3594:
-
Summary: When Group by Partition Column Type is Timestamp or STRING Which
Format contains HH:MM:SS, It will occur URISyntaxException (was: When Group
by Partition column type is Timestamp or STRING which format contains
HH:MM:SS, It will occur URISyntaxException)
When Group by Partition Column Type is Timestamp or STRING Which Format
contains HH:MM:SS, It will occur URISyntaxException
-
Key: HIVE-3594
URL: https://issues.apache.org/jira/browse/HIVE-3594
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Reporter: Daisy.Yuan
create table test (no int, name string) partitioned by (pts string) row
format delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts2.txt' into table test
partition(pts='21:25:12');
load data local inpath '/opt/files/groupbyts3.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts4.txt' into table test
partition(pts='21:25:12');
when I execute “select * from test group by pts;”, it will occur as follows
exception.
at org.apache.hadoop.fs.Path.initialize(Path.java:157)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.exec.Utilities.getInputSummary(Utilities.java:1667)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.estimateNumberOfReducers(MapRedTask.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.setNumberOfReducers(MapRedTask.java:400)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:93)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1329)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:954)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
fake-path-metadata-only-query-default.test{pts=12:11:30%7D
at java.net.URI.checkPath(URI.java:1788)
at java.net.URI.init(URI.java:734)
at org.apache.hadoop.fs.Path.initialize(Path.java:154)
... 19 more
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
When PhysicalOptimizer optimizes GroupByOperator, according to default
parameters hive.optimize.metadataonly = true, MetadataOnlyOptimizer will be
enabled. The MetadataOnlyOptimizer will change the partition alias desc. The
partition alies
hdfs://160.165.0.1:9000/user/hive/warehouse/test/pts=12%3A11%3A30 is
changed into
fake-path-metadata-only-query-default.test{pts=12:11:30}. When construct uri
through new partition alies, it must occur java.net.URISyntaxException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3589) describe/show partition/show tblproperties command should accept database name
[ https://issues.apache.org/jira/browse/HIVE-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3589: -- Attachment: HIVE-3589.D6075.1.patch navis requested code review of HIVE-3589 [jira] describe/show partition/show tblproperties command should accept database name. Reviewers: JIRA DPAL-1916 describe/show partition/show tblproperties command should accept database name describe command not giving the details when called as describe dbname.tablename. Throwing the error Table dbname not found. Ex: hive -e describe masterdb.table1 will throw error Table masterdb not found TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D6075 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/ShowPartitionsDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/ShowTblPropertiesDesc.java ql/src/test/queries/clientpositive/describe_table.q ql/src/test/queries/clientpositive/show_partitions.q ql/src/test/queries/clientpositive/show_tblproperties.q ql/src/test/results/clientpositive/describe_table.q.out ql/src/test/results/clientpositive/show_partitions.q.out ql/src/test/results/clientpositive/show_tblproperties.q.out MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/14523/ To: JIRA, navis describe/show partition/show tblproperties command should accept database name -- Key: HIVE-3589 URL: https://issues.apache.org/jira/browse/HIVE-3589 Project: Hive Issue Type: Bug Components: Metastore, Query Processor Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Assignee: Navis Priority: Minor Attachments: HIVE-3589.D6075.1.patch describe command not giving the details when called as describe dbname.tablename. Throwing the error Table dbname not found. Ex: hive -e describe masterdb.table1 will throw error Table masterdb not found -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3589) describe/show partition/show tblproperties command should accept database name
[ https://issues.apache.org/jira/browse/HIVE-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-3589: Status: Patch Available (was: Open) describe/show partition/show tblproperties command should accept database name -- Key: HIVE-3589 URL: https://issues.apache.org/jira/browse/HIVE-3589 Project: Hive Issue Type: Bug Components: Metastore, Query Processor Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Assignee: Navis Priority: Minor Attachments: HIVE-3589.D6075.1.patch describe command not giving the details when called as describe dbname.tablename. Throwing the error Table dbname not found. Ex: hive -e describe masterdb.table1 will throw error Table masterdb not found -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3209) QTestUtil should filter out only env-dependent part of paths from explain result
[ https://issues.apache.org/jira/browse/HIVE-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis resolved HIVE-3209. - Resolution: Duplicate HIVE-3556 QTestUtil should filter out only env-dependent part of paths from explain result Key: HIVE-3209 URL: https://issues.apache.org/jira/browse/HIVE-3209 Project: Hive Issue Type: Improvement Components: Testing Infrastructure Affects Versions: 0.10.0 Environment: ubuntu 10.04 Reporter: Navis Assignee: Navis Priority: Trivial Bucket mapping information is always masked. But it could be crucial for checking regressions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3594) When Group by Partition Column Type is Timestamp or STRING Which Format contains HH:MM:SS, It will occur URISyntaxException
[
https://issues.apache.org/jira/browse/HIVE-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-3594:
--
Attachment: HIVE-3594.D6081.1.patch
navis requested code review of HIVE-3594 [jira] When Group by Partition Column
Type is Timestamp or STRING Which Format contains HH:MM:SS, It will occur
URISyntaxException.
Reviewers: JIRA
DPAL-1917 When Group by Partition Column Type is Timestamp or STRING Which
Format contains HH:MM:SS, It will occur URISyntaxException
create table test (no int, name string) partitioned by (pts string) row
format delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts2.txt' into table test
partition(pts='21:25:12');
load data local inpath '/opt/files/groupbyts3.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts4.txt' into table test
partition(pts='21:25:12');
when I execute “select * from test group by pts;”, it will occur as follows
exception.
at org.apache.hadoop.fs.Path.initialize(Path.java:157)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.exec.Utilities.getInputSummary(Utilities.java:1667)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.estimateNumberOfReducers(MapRedTask.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.setNumberOfReducers(MapRedTask.java:400)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:93)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1329)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:954)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
fake-path-metadata-only-query-default.test{pts=12:11:30%7D
at java.net.URI.checkPath(URI.java:1788)
at java.net.URI.init(URI.java:734)
at org.apache.hadoop.fs.Path.initialize(Path.java:154)
... 19 more
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
When PhysicalOptimizer optimizes GroupByOperator, according to default
parameters hive.optimize.metadataonly = true, MetadataOnlyOptimizer will be
enabled. The MetadataOnlyOptimizer will change the partition alias desc. The
partition alies
hdfs://160.165.0.1:9000/user/hive/warehouse/test/pts=12%3A11%3A30 is changed
into
fake-path-metadata-only-query-default.test{pts=12:11:30}. When construct uri
through new partition alies, it must occur java.net.URISyntaxException.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D6081
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MetadataOnlyOptimizer.java
ql/src/test/queries/clientpositive/metadataonly1.q
ql/src/test/results/clientpositive/metadataonly1.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/14529/
To: JIRA, navis
When Group by Partition Column Type is Timestamp or STRING Which Format
contains HH:MM:SS, It will occur URISyntaxException
-
Key: HIVE-3594
URL: https://issues.apache.org/jira/browse/HIVE-3594
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Reporter: Daisy.Yuan
Attachments: HIVE-3594.D6081.1.patch
create table test (no int, name string) partitioned by (pts string) row
format delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
[jira] [Assigned] (HIVE-3594) When Group by Partition Column Type is Timestamp or STRING Which Format contains HH:MM:SS, It will occur URISyntaxException
[
https://issues.apache.org/jira/browse/HIVE-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis reassigned HIVE-3594:
---
Assignee: Navis
When Group by Partition Column Type is Timestamp or STRING Which Format
contains HH:MM:SS, It will occur URISyntaxException
-
Key: HIVE-3594
URL: https://issues.apache.org/jira/browse/HIVE-3594
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Reporter: Daisy.Yuan
Assignee: Navis
Attachments: HIVE-3594.D6081.1.patch
create table test (no int, name string) partitioned by (pts string) row
format delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts2.txt' into table test
partition(pts='21:25:12');
load data local inpath '/opt/files/groupbyts3.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts4.txt' into table test
partition(pts='21:25:12');
when I execute “select * from test group by pts;”, it will occur as follows
exception.
at org.apache.hadoop.fs.Path.initialize(Path.java:157)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.exec.Utilities.getInputSummary(Utilities.java:1667)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.estimateNumberOfReducers(MapRedTask.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.setNumberOfReducers(MapRedTask.java:400)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:93)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1329)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:954)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
fake-path-metadata-only-query-default.test{pts=12:11:30%7D
at java.net.URI.checkPath(URI.java:1788)
at java.net.URI.init(URI.java:734)
at org.apache.hadoop.fs.Path.initialize(Path.java:154)
... 19 more
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
When PhysicalOptimizer optimizes GroupByOperator, according to default
parameters hive.optimize.metadataonly = true, MetadataOnlyOptimizer will be
enabled. The MetadataOnlyOptimizer will change the partition alias desc. The
partition alies
hdfs://160.165.0.1:9000/user/hive/warehouse/test/pts=12%3A11%3A30 is
changed into
fake-path-metadata-only-query-default.test{pts=12:11:30}. When construct uri
through new partition alies, it must occur java.net.URISyntaxException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3594) When Group by Partition Column Type is Timestamp or STRING Which Format contains HH:MM:SS, It will occur URISyntaxException
[
https://issues.apache.org/jira/browse/HIVE-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daisy.Yuan updated HIVE-3594:
-
Description:
create table test (no int, name string) partitioned by (pts string) row format
delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts2.txt' into table test
partition(pts='21:25:12');
load data local inpath '/opt/files/groupbyts3.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts4.txt' into table test
partition(pts='21:25:12');
when I execute “select * from test group by pts;”, it will occur as follows
exception.
at org.apache.hadoop.fs.Path.initialize(Path.java:157)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.exec.Utilities.getInputSummary(Utilities.java:1667)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.estimateNumberOfReducers(MapRedTask.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.setNumberOfReducers(MapRedTask.java:400)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:93)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1329)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:954)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
fake-path-metadata-only-query-default.test{pts=12:11:30%7D
at java.net.URI.checkPath(URI.java:1788)
at java.net.URI.init(URI.java:734)
at org.apache.hadoop.fs.Path.initialize(Path.java:154)
... 19 more
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
When PhysicalOptimizer optimizes GroupByOperator, according to default
parameters hive.optimize.metadataonly = true, MetadataOnlyOptimizer will be
enabled. The MetadataOnlyOptimizer will change the partition alias desc. The
partition alies hdfs://ip:9000/user/hive/warehouse/test/pts=12%3A11%3A30 is
changed into
fake-path-metadata-only-query-default.test{pts=12:11:30}. When construct uri
through new partition alies, it must occur java.net.URISyntaxException.
was:
create table test (no int, name string) partitioned by (pts string) row format
delimited fields terminated by ' ';
load data local inpath '/opt/files/groupbyts1.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts2.txt' into table test
partition(pts='21:25:12');
load data local inpath '/opt/files/groupbyts3.txt' into table test
partition(pts='12:11:30');
load data local inpath '/opt/files/groupbyts4.txt' into table test
partition(pts='21:25:12');
when I execute “select * from test group by pts;”, it will occur as follows
exception.
at org.apache.hadoop.fs.Path.initialize(Path.java:157)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.exec.Utilities.getInputSummary(Utilities.java:1667)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.estimateNumberOfReducers(MapRedTask.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.setNumberOfReducers(MapRedTask.java:400)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:93)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1329)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:954)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630)
at
[jira] [Commented] (HIVE-3523) Hive info logging is broken
[ https://issues.apache.org/jira/browse/HIVE-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478644#comment-13478644 ] Ashutosh Chauhan commented on HIVE-3523: +1 Please commit if tests pass. Hive info logging is broken --- Key: HIVE-3523 URL: https://issues.apache.org/jira/browse/HIVE-3523 Project: Hive Issue Type: Bug Components: Logging Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Carl Steinbach Attachments: HIVE-3523.1.patch.txt, HIVE-3523.D5811.1.patch Hive Info logging is broken on trunk. hive -hiveconf hive.root.logger=INFO,console doesn't print the output of LOG.info statements to the console. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3571) add a way to run a small unit quickly
[ https://issues.apache.org/jira/browse/HIVE-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-3571: Attachment: HIVE-3571.1.patch.txt ant test-ql -Dtestcase=TestCliDriver -Dqfile=union.q I know little about ant script. Someone can generify this for common usage (it's only for tests in ql module) add a way to run a small unit quickly - Key: HIVE-3571 URL: https://issues.apache.org/jira/browse/HIVE-3571 Project: Hive Issue Type: Test Components: Testing Infrastructure Reporter: Namit Jain Attachments: HIVE-3571.1.patch.txt A simple unit test: ant test -Dtestcase=TestCliDriver -Dqfile=groupby2.q takes a long time. There should be a quick way to achieve that for debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
