[jira] [Updated] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2503: Attachment: HIVE-2503.1.patch.txt Removed diff-prefix and renamed patch file as supposed to be. > HiveServer should provide per session configuration > --- > > Key: HIVE-2503 > URL: https://issues.apache.org/jira/browse/HIVE-2503 > Project: Hive > Issue Type: Bug > Components: CLI, Server Infrastructure >Reporter: Navis > Attachments: HIVE-2503.1.patch.txt > > > Currently ThriftHiveProcessorFactory returns same HiveConf instance to > HiveServerHandler, making impossible to use per sesssion configuration. Just > wrapping 'conf' -> 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-2503: create per-session HiveConf instance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ --- Review request for hive and Carl Steinbach. Summary --- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503. https://issues.apache.org/jira/browse/HIVE-2503 Diffs - service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f Diff: https://reviews.apache.org/r/2820/diff Testing --- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
[jira] [Commented] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149504#comment-13149504 ] jirapos...@reviews.apache.org commented on HIVE-2503: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ --- Review request for hive and Carl Steinbach. Summary --- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503. https://issues.apache.org/jira/browse/HIVE-2503 Diffs - service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f Diff: https://reviews.apache.org/r/2820/diff Testing --- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis > HiveServer should provide per session configuration > --- > > Key: HIVE-2503 > URL: https://issues.apache.org/jira/browse/HIVE-2503 > Project: Hive > Issue Type: Bug > Components: CLI, Server Infrastructure >Reporter: Navis > Attachments: HIVE-2503.1.patch.txt > > > Currently ThriftHiveProcessorFactory returns same HiveConf instance to > HiveServerHandler, making impossible to use per sesssion configuration. Just > wrapping 'conf' -> 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1083 - Fixed
Changes for Build #1081 Changes for Build #1082 [namit] HIVE-2572 HIVE-2246 upgrade script changed the COLUMNS_V2.COMMENT length (Ning Zhang via namit) Changes for Build #1083 [nzhang] HIVE-2553. Introduction of Hashing for IN operator for constant values (Robert Surówka via Ning Zhang) All tests passed The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1083) Status: Fixed Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1083/ to view the results.
[jira] [Commented] (HIVE-2553) Use hashing instead of list traversal for IN operator for primitive types
[ https://issues.apache.org/jira/browse/HIVE-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149505#comment-13149505 ] Hudson commented on HIVE-2553: -- Integrated in Hive-trunk-h0.21 #1083 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1083/]) HIVE-2553. Introduction of Hashing for IN operator for constant values (Robert Surówka via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201600 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIn.java * /hive/trunk/ql/src/test/queries/clientpositive/input49.q * /hive/trunk/ql/src/test/results/clientpositive/input49.q.out * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/StructTypeInfo.java > Use hashing instead of list traversal for IN operator for primitive types > - > > Key: HIVE-2553 > URL: https://issues.apache.org/jira/browse/HIVE-2553 > Project: Hive > Issue Type: Improvement >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Fix For: 0.9.0 > > Attachments: HIVE-2553.1.patch, HIVE-2553.2.patch, HIVE-2553.3.patch, > HIVE-2553.4.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2573) Create per-session function registry
[
https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2573:
Attachment: HIVE-2573.1.patch.txt
This patch induces sentences like this
{code}
CREATE [SESSION] TEMPORARY FUNCTION ...
DROP [SESSION] TEMPORARY FUNCTION ...
{code}
I've thought keyword 'TEMPORARY' could mean the 'session' thing but it seemed
to be not safe.
> Create per-session function registry
> -
>
> Key: HIVE-2573
> URL: https://issues.apache.org/jira/browse/HIVE-2573
> Project: Hive
> Issue Type: Improvement
> Components: Server Infrastructure
>Reporter: Navis
>Priority: Minor
> Attachments: HIVE-2573.1.patch.txt
>
>
> Currently the function registry is shared resource and could be overrided by
> other users when using HiveServer. If per-session function registry is
> provided, this situation could be prevented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2576) HiveDataSource doesn't get a proper connection
[ https://issues.apache.org/jira/browse/HIVE-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Lalevée updated HIVE-2576: -- Attachment: HIVE-2576-r1201637.patch Here is the trivial patch > HiveDataSource doesn't get a proper connection > -- > > Key: HIVE-2576 > URL: https://issues.apache.org/jira/browse/HIVE-2576 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Nicolas Lalevée > Attachments: HIVE-2576-r1201637.patch > > > The HiveDataSource is creating a HiveConnection with as an URL "", but the > connection expects to start with "jdbc:hive://" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2576) HiveDataSource doesn't get a proper connection
HiveDataSource doesn't get a proper connection -- Key: HIVE-2576 URL: https://issues.apache.org/jira/browse/HIVE-2576 Project: Hive Issue Type: Bug Components: JDBC Reporter: Nicolas Lalevée Attachments: HIVE-2576-r1201637.patch The HiveDataSource is creating a HiveConnection with as an URL "", but the connection expects to start with "jdbc:hive://" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2577) Expose the HiveConf in HiveConnection API
Expose the HiveConf in HiveConnection API - Key: HIVE-2577 URL: https://issues.apache.org/jira/browse/HIVE-2577 Project: Hive Issue Type: Improvement Reporter: Nicolas Lalevée Attachments: HIVE-2577-r1201637.patch When running the jdbc code in a local mode, there no way to programatically manage the hive conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2577) Expose the HiveConf in HiveConnection API
[ https://issues.apache.org/jira/browse/HIVE-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Lalevée updated HIVE-2577: -- Attachment: HIVE-2577-r1201637.patch Here is the trivial patch > Expose the HiveConf in HiveConnection API > - > > Key: HIVE-2577 > URL: https://issues.apache.org/jira/browse/HIVE-2577 > Project: Hive > Issue Type: Improvement >Reporter: Nicolas Lalevée > Attachments: HIVE-2577-r1201637.patch > > > When running the jdbc code in a local mode, there no way to programatically > manage the hive conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149558#comment-13149558 ] Bennie Schut commented on HIVE-2573: This probably fixes HIVE-2544 also. That would be nice. > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2235) Getting an SQLException when requesting Metadata about column 5 of Connection.getColumns()
[ https://issues.apache.org/jira/browse/HIVE-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149635#comment-13149635 ] Priyadarshini commented on HIVE-2235: - For repro3.java, I have verified with the trunk code. After getting the ColumnType from the ResultSetMetaData, i am not getting any SQL Exception. > Getting an SQLException when requesting Metadata about column 5 of > Connection.getColumns() > -- > > Key: HIVE-2235 > URL: https://issues.apache.org/jira/browse/HIVE-2235 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 0.7.0 > Environment: Connecting from 32bit Windows via JDBC to a Hive > installation on Linux Kernel-2.6.18-glibc-2.5 x86_64 64bit >Reporter: Alex Piyevsky > Attachments: repro3.class, repro3.java, repro3.log > > > Getting an SQLException when requesting Metadata about column 5 of > Connection.getColumns() answer set. > Failure occurs when invoking DatabaseMetaData.getColumnType(5), resulting in: > java.sql.SQLException: Unrecognized column type: I32 > Repro materials are attached, please let me know if more info is required > Here is the log that the repro generates: > BUG SYNOPSIS: Getting error > "Unrecognized column type: I32" > returned by > DatabaseMetadata.getColumnType(5) method call > * > DriverManager.getConnection > Back from connecting > Executing DatabaseMetaData.getColumns(null,null,"sergio",null) > Print column types for columns of getColumns() answer set > Column Type for column 1 is 12 > Column Type for column 2 is 12 > Column Type for column 3 is 12 > Column Type for column 4 is 12 > *** SQLException caught *** > SQLState: null > Message: Unrecognized column type: I32 > Vendor: 0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2467) HA Support for Metastore Server
[ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149742#comment-13149742 ] Thomas Weise commented on HIVE-2467: Yes, this was the intention. Also, the current ZK token store implementation could be translated into a SQL implementation with relatively small effort (it does not depend on ZK watches for invalidation). For our use, we plan to deal with the security limitation at L3 and use ZK security when it becomes available. Longer term, assuming there will be use cases outside HCatalog/Hive for long running processes that require token support in conjunction with HA but without HA RDBMS already available, a ZK based token storage might be a more likely choice. While plain HDFS is secure it does not support the access pattern needed here (fast random reads and writes). > HA Support for Metastore Server > > > Key: HIVE-2467 > URL: https://issues.apache.org/jira/browse/HIVE-2467 > Project: Hive > Issue Type: Improvement > Components: Metastore, Security, Server Infrastructure >Affects Versions: 0.8.0, 0.9.0 >Reporter: Thomas Weise >Assignee: Thomas Weise > Fix For: 0.9.0 > > Attachments: HIVE-2467.patch > > > We require HA deployment for metastore server for HCatalog: > * Multiple server instances run behind VIP > * Database provides HA > Metastore server instances will need to be able to share any state required > for VIP outside RDBMS. As of Hive 0.8 affected conversational state that > needs to support VIP/HA setup is limited to current delegation tokens. Is > this correct? > We are planning to use ZooKeeper to share current delegation tokens and > master keys between nodes of the VIP. ZK is already (optionally) used by Hive > for concurrency control. Access to ZK would be limited on the network level > or in the future, when ZooKeeper supports security, through Kerberos, similar > to NN access. > Currently Hive taps into Hadoop core security delegation token support > through extension of > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager > A solution could amend the Hive specific extension to support: > * Pluggable delegation token and master key store (ZooKeeper as alternative > for in-memory AbstractDelegationTokenSecretManager) > * Delegation token retrieval from token store when not found in memory > (wrap/extend retrievePassword(...)) > * Cancellation of token in token store > * Purging of expired tokens from token store > http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2571) eclipse template .classpath is broken
[ https://issues.apache.org/jira/browse/HIVE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2571: - Priority: Minor (was: Major) > eclipse template .classpath is broken > - > > Key: HIVE-2571 > URL: https://issues.apache.org/jira/browse/HIVE-2571 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Fix For: 0.9.0 > > Attachments: HIVE-2571.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2571) eclipse template .classpath is broken
[ https://issues.apache.org/jira/browse/HIVE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149766#comment-13149766 ] jirapos...@reviews.apache.org commented on HIVE-2571: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2800/ --- (Updated 2011-11-14 17:31:02.776950) Review request for Ning Zhang and namit jain. Changes --- This patch should will make the eclipse classpath point to places where needed jars are available from the very beginning. Apparently, even without this change the setup described here: https://cwiki.apache.org/confluence/display/Hive/GettingStarted+EclipseSetup works, yet with this patch it should work also with just ant jar instead of ant package. Summary --- HIVE-2571 update eclipse classpath template This addresses bug HIVE-2571. https://issues.apache.org/jira/browse/HIVE-2571 Diffs (updated) - trunk/eclipse-templates/.classpath 1201784 Diff: https://reviews.apache.org/r/2800/diff Testing --- project build with eclipse. Thanks, Robert > eclipse template .classpath is broken > - > > Key: HIVE-2571 > URL: https://issues.apache.org/jira/browse/HIVE-2571 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Fix For: 0.9.0 > > Attachments: HIVE-2571.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2571) eclipse template .classpath is broken
[ https://issues.apache.org/jira/browse/HIVE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2571: - Attachment: HIVE-2571.2.patch > eclipse template .classpath is broken > - > > Key: HIVE-2571 > URL: https://issues.apache.org/jira/browse/HIVE-2571 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Fix For: 0.9.0 > > Attachments: HIVE-2571.1.patch, HIVE-2571.2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2567) Some CTAS queries with * and group by don't work.
[ https://issues.apache.org/jira/browse/HIVE-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149791#comment-13149791 ] jirapos...@reviews.apache.org commented on HIVE-2567: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2792/ --- (Updated 2011-11-14 18:06:11.336387) Review request for Ning Zhang and namit jain. Changes --- Updated the patch to apply cleanly on current trunk Summary --- To introduce proper support for the problem would require a lot of work I believe. Yet this patch seems to do the job pretty well, and with very good chance doesn't break anything :). When user puts tablename.columnname in group by, then the name of the column from * won't be columname but tablename_columnname, to allow ctas with * and joins on tables that share column names (like in the test). In very rare cases it may lead to duplicate column anyway, as shown in the negative test. This isn't the final resolution of the problem (as that would require substantial changes), yet at least this allows hive to support many use cases with ctas, * and group by. I also run all the tests (with -overwrite option) for this change, the tests executed correctly and no output file for any already existing test had been changed. This addresses bug HIVE-2567. https://issues.apache.org/jira/browse/HIVE-2567 Diffs (updated) - trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1201807 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1201807 trunk/ql/src/test/queries/clientnegative/ctas_group_by_failure1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/input48.q PRE-CREATION trunk/ql/src/test/results/clientnegative/ctas_group_by_failure1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/input48.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2792/diff Testing --- Worked on some sample queries. Unit tests work too. Thanks, Robert > Some CTAS queries with * and group by don't work. > - > > Key: HIVE-2567 > URL: https://issues.apache.org/jira/browse/HIVE-2567 > Project: Hive > Issue Type: Bug >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Attachments: HIVE-2567.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2578) Debug mode in some situations doesn't work properly when child JVM is started from MapRedLocalTask
Debug mode in some situations doesn't work properly when child JVM is started from MapRedLocalTask --- Key: HIVE-2578 URL: https://issues.apache.org/jira/browse/HIVE-2578 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: Use sorted nature of compact indexes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2605/ --- (Updated 2011-11-14 20:10:27.544678) Review request for hive, Yongqiang He, Ning Zhang, and namit jain. Changes --- This diff is in response to Yongqiang's comments. He pointed out that if the ExprNode in a FilterOperator is a tree, rather than a single operator, the optimizations will not be applied. This was fixed by marking the FilterOperator, and the ExprNode in the tree, where the index column is compared. When the FilterOperator needs to execute a comparison, it traverses the tree of ExprNodes until it identifies the marked ExprNode which has the context necessary to perform the comparison. There was also an issue where CombineHiveRecordReaders were not always being marked as sorted when they should. This was fixed by marking them as sorted as part of the initIOContext call, instead of as part of the HiveContextAwareRecordReader constructor. Instead of determining if the input format class supports making use of sorted input by string comparison of the class name in the CompactIndexHandler, I check whether or not the class extends HiveInputFormat. Currently this is sufficient as all of the InputFormats introduced in Hive extend this, and I ensured that all such InputFormats do indeed support making use of sorted input. Summary --- The CompactIndexHandler determines if the reentrant query it creates is a candidate for using the fact the index is sorted (it has an appropriate number of non-partition conditions, and the query plan is of the form expected). It sets the input format to HiveSortedInputFormat, and marks the FilterOperator for the non-partition condition. The HiveSortedInputFormat is extends HiveInputFormat, so its splits consist of data from a single file, and its record reader is HiveBinarySearchRecordReader. HiveBinarySearchRecordReader starts by assuming it is performing a binary search. It sets the appropriate flags in IOContext, which acts as the means of communication between the FilterOperators and the record reader. The non-partition FilterOperator is responsible for executing a comparison between the value in the row and column of interest and the constant. It also provides the type of the generic UDF. It sets this data in the IOContext. As long as the binary search continues the FilterOperators do not forward rows to the operators below them. The record reader uses the comparison and the type of the generic UDF to execute a binary search on the underlying RCFile until it finds the block of interest, or determines that if any block is of interest it is the last one. The search then proceeds linearly from the beginning of the identified block. If ever in the binary search a problem occurs, like the comparison fails for some reason, a linear search begins from the beginning of the data which has yet to be eliminated. Regardless of whether or not a binary search is performed, the record reader attempts to end the linear search as soon as it can based on the comparison and the type of the generic UDF. This addresses bug HIVE-2535. https://issues.apache.org/jira/browse/HIVE-2535 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1183507 trunk/conf/hive-default.xml 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFileRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseCompare.java 1183507 trunk/ql/src/test/org/apache/hadoop/hive/ql/hooks/VerifyHiveSortedInputFormatUse
[jira] [Updated] (HIVE-2578) Debug mode in some situations doesn't work properly when child JVM is started from MapRedLocalTask
[ https://issues.apache.org/jira/browse/HIVE-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2578: - Attachment: HIVE-2578.1.patch > Debug mode in some situations doesn't work properly when child JVM is > started from MapRedLocalTask > --- > > Key: HIVE-2578 > URL: https://issues.apache.org/jira/browse/HIVE-2578 > Project: Hive > Issue Type: Bug >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Attachments: HIVE-2578.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2578) Debug mode in some situations doesn't work properly when child JVM is started from MapRedLocalTask
[ https://issues.apache.org/jira/browse/HIVE-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2578: - Status: Patch Available (was: Open) > Debug mode in some situations doesn't work properly when child JVM is > started from MapRedLocalTask > --- > > Key: HIVE-2578 > URL: https://issues.apache.org/jira/browse/HIVE-2578 > Project: Hive > Issue Type: Bug >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Attachments: HIVE-2578.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2535) Use sorted nature of compact indexes
[ https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149864#comment-13149864 ] jirapos...@reviews.apache.org commented on HIVE-2535: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2605/ --- (Updated 2011-11-14 20:10:27.544678) Review request for hive, Yongqiang He, Ning Zhang, and namit jain. Changes --- This diff is in response to Yongqiang's comments. He pointed out that if the ExprNode in a FilterOperator is a tree, rather than a single operator, the optimizations will not be applied. This was fixed by marking the FilterOperator, and the ExprNode in the tree, where the index column is compared. When the FilterOperator needs to execute a comparison, it traverses the tree of ExprNodes until it identifies the marked ExprNode which has the context necessary to perform the comparison. There was also an issue where CombineHiveRecordReaders were not always being marked as sorted when they should. This was fixed by marking them as sorted as part of the initIOContext call, instead of as part of the HiveContextAwareRecordReader constructor. Instead of determining if the input format class supports making use of sorted input by string comparison of the class name in the CompactIndexHandler, I check whether or not the class extends HiveInputFormat. Currently this is sufficient as all of the InputFormats introduced in Hive extend this, and I ensured that all such InputFormats do indeed support making use of sorted input. Summary --- The CompactIndexHandler determines if the reentrant query it creates is a candidate for using the fact the index is sorted (it has an appropriate number of non-partition conditions, and the query plan is of the form expected). It sets the input format to HiveSortedInputFormat, and marks the FilterOperator for the non-partition condition. The HiveSortedInputFormat is extends HiveInputFormat, so its splits consist of data from a single file, and its record reader is HiveBinarySearchRecordReader. HiveBinarySearchRecordReader starts by assuming it is performing a binary search. It sets the appropriate flags in IOContext, which acts as the means of communication between the FilterOperators and the record reader. The non-partition FilterOperator is responsible for executing a comparison between the value in the row and column of interest and the constant. It also provides the type of the generic UDF. It sets this data in the IOContext. As long as the binary search continues the FilterOperators do not forward rows to the operators below them. The record reader uses the comparison and the type of the generic UDF to execute a binary search on the underlying RCFile until it finds the block of interest, or determines that if any block is of interest it is the last one. The search then proceeds linearly from the beginning of the identified block. If ever in the binary search a problem occurs, like the comparison fails for some reason, a linear search begins from the beginning of the data which has yet to be eliminated. Regardless of whether or not a binary search is performed, the record reader attempts to end the linear search as soon as it can based on the comparison and the type of the generic UDF. This addresses bug HIVE-2535. https://issues.apache.org/jira/browse/HIVE-2535 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1183507 trunk/conf/hive-default.xml 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFileRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/pl
[jira] [Commented] (HIVE-2578) Debug mode in some situations doesn't work properly when child JVM is started from MapRedLocalTask
[ https://issues.apache.org/jira/browse/HIVE-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149865#comment-13149865 ] jirapos...@reviews.apache.org commented on HIVE-2578: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2825/ --- Review request for Ning Zhang. Summary --- Basically there is an additional place in Hive where a child JVM can be started, that I wasn't aware of. I added to that place (MapRedLocalTask) update of debug environment variables as it is being already done in MapRedTask. By the way I added new debug option: swapSuspend, since I was very often using hive --debug:mainSuspend=n,childSuspend=y whereas now --debug:swapSuspend will be sufficient. This addresses bug HIVE-2578. https://issues.apache.org/jira/browse/HIVE-2578 Diffs - trunk/bin/ext/debug.sh 1201858 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java 1201858 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 1201858 Diff: https://reviews.apache.org/r/2825/diff Testing --- Tested on sample queries that it works (e.g. select s1.key, count(*) from src s1 join src s2 on s1.key = s2.key group by s1.key, s2.value; ) Thanks, Robert > Debug mode in some situations doesn't work properly when child JVM is > started from MapRedLocalTask > --- > > Key: HIVE-2578 > URL: https://issues.apache.org/jira/browse/HIVE-2578 > Project: Hive > Issue Type: Bug >Reporter: Robert Surówka >Assignee: Robert Surówka >Priority: Minor > Attachments: HIVE-2578.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2535) Use sorted nature of compact indexes
[ https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2535: Attachment: HIVE-2535.4.patch.txt > Use sorted nature of compact indexes > > > Key: HIVE-2535 > URL: https://issues.apache.org/jira/browse/HIVE-2535 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2535.1.patch.txt, HIVE-2535.2.patch.txt, > HIVE-2535.3.patch.txt, HIVE-2535.4.patch.txt > > > Compact indexes are sorted based on the indexed columns, but we are not using > this fact when we access the index. > To start with, if the index is stored as an RC file, and if the predicate > being used to access the index consists of only one non-partition condition > using one of the operators >,>=,<,<=,= we could use a binary search (if > necessary) to find the block to begin scanning for unfiltered rows, and we > could use the result of comparing the value in the column with the constant > (this is necessarily the form of a predicate which is optimized using an > index) to determine when we have found all the rows which will be unfiltered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[ANNOUNCE] Next Hive Contributors Meeting December 5th @ Facebook
Hi, The next Hive Contributors Meeting will convene on Monday, December 5th at 4pm at Facebook's office in Palo Alto. More information about the event including an updated agenda can be found on the meetup page: http://www.meetup.com/Hive-Contributors-Group/events/41150912/ Please RSVP if you plan to attend. Thanks. Carl
[jira] [Commented] (HIVE-2315) DatabaseMetadata.getColumns() does not return partition column names for a table
[
https://issues.apache.org/jira/browse/HIVE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149933#comment-13149933
]
Doug Sedlak commented on HIVE-2315:
---
For an 'int' partition column, the 'getColumnTypeName' metadata call is
returning type 'string'. However, a DESCRIBE correctly returns type 'int'.
Seems like a bug if it replicates for others? I have the patched code in place
on both client and server. Thanks!
Sample table is:
create EXTERNAL table partition_by_int (str_val string) PARTITIONED BY (int_col
INT) ROW FORMAT delimited fields terminated by ',' STORED AS TEXTFILE;
And (shortened) JDBC code is:
res = stmt.executeQuery("select * from partition_by_int");
ResultSetMetaData metaData = res.getMetaData();
for (int i = 0; i < metaData.getColumnCount(); i++) {
System.out.print(metaData.getColumnName(i + 1) + " \t");
System.out.print(metaData.getColumnDisplaySize(i + 1) + "\t");
System.out.println(metaData.getColumnTypeName(i + 1));
}
With output:
str_val 2147483647 string
int_col 2147483647 string
In contrast, DESCRIBE partition_by_int generates:
str_val string
int_col int
> DatabaseMetadata.getColumns() does not return partition column names for a
> table
>
>
> Key: HIVE-2315
> URL: https://issues.apache.org/jira/browse/HIVE-2315
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 0.7.1
>Reporter: Mythili Gopalakrishnan
>Assignee: Patrick Hunt
>Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2315.patch, HIVE-2315.patch
>
>
> getColumns() method of DatabaseMetadata for HIVE JDBC Driver does not return
> the partition column names. Where as from HIVE CLI, if you do a 'describe
> tablename' you get all columns including the partition columns. It would be
> nice if getColumns() method returns all columns.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2566) reduce the number map-reduce jobs for union all
[ https://issues.apache.org/jira/browse/HIVE-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150013#comment-13150013 ] He Yongqiang commented on HIVE-2566: looks good, running tests. > reduce the number map-reduce jobs for union all > --- > > Key: HIVE-2566 > URL: https://issues.apache.org/jira/browse/HIVE-2566 > Project: Hive > Issue Type: Improvement >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: HIVE-2566.D405.1.patch, HIVE-2566.D405.2.patch > > > A query like: > select s.key, s.value from ( > select key, value from src2 where key < 10 > union all > select key, value from src3 where key < 10 > union all > select key, value from src4 where key < 10 > union all > select key, count(1) as value from src5 group by key > )s; > should run the last sub-query > 'select key, count(1) as value from src5 group by key' > as a map-reduce job. > And then the union should be a map-only job reading from the first 3 map-only > subqueries > and the output of the last map-reduce job. > The current plan is very inefficient. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2570) ShimLoader does not work with Hadoop 0.20.20#.0 because of nano version.
[ https://issues.apache.org/jira/browse/HIVE-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur resolved HIVE-2570. -- Resolution: Invalid My bad, read the code wrong. > ShimLoader does not work with Hadoop 0.20.20#.0 because of nano version. > > > Key: HIVE-2570 > URL: https://issues.apache.org/jira/browse/HIVE-2570 > Project: Hive > Issue Type: Bug >Affects Versions: 0.7.1, 0.9.0 >Reporter: Alejandro Abdelnur >Priority: Critical > > The ShimsLoader getMajorVersion() method expects a version is #.#.# (2 dots > only) but the Hadoop security releases are #.#.#.# (ie 0.20.205.0) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2391) published POMs in Maven repo are incorrect
[ https://issues.apache.org/jira/browse/HIVE-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150081#comment-13150081 ] Alejandro Abdelnur commented on HIVE-2391: -- Adding 'hive-contrib' as an explicit dependency fixes things and I'm able to use Hive 0.9.0-SNAPSHOT, THXS! Still, hive POMs refer to an invalid dependency, groupId=hadoop & artifactId=core. Iit should be 'org.apache.hadoop' and 'hadoop-core' respectively. > published POMs in Maven repo are incorrect > -- > > Key: HIVE-2391 > URL: https://issues.apache.org/jira/browse/HIVE-2391 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.7.1 >Reporter: Alejandro Abdelnur >Assignee: Carl Steinbach >Priority: Critical > Fix For: 0.8.0 > > Attachments: HIVE-2391.1.patch.txt, HIVE-2391.2.patch.txt, > HIVE-2391.3.patch.txt, HIVE-2391.4.patch.txt, HIVE-2391.5.patch.txt, > HIVE-2391.wip.1.patch.txt > > > The Hive artifacts published in Apache Maven SNAPSHOTS repo are incorrect. > Dependencies are not complete. > Even after adding as dependencies ALL the Hive artifacts it is not possible > to compile a project using Hive JARs (I'm trying to integrate Oozie Hive > Action using Apache Hive). > As a reference the Hive CDH POMs dependencies could be used (Using those > artifacts I'm able to compile/test/run Hive from within Oozie). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2556) upgrade script 008-HIVE-2246.mysql.sql contains syntax errors
[ https://issues.apache.org/jira/browse/HIVE-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2556: - Component/s: Metastore > upgrade script 008-HIVE-2246.mysql.sql contains syntax errors > - > > Key: HIVE-2556 > URL: https://issues.apache.org/jira/browse/HIVE-2556 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.8.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.8.0, 0.9.0 > > Attachments: D309.1.patch, HIVE-2556.patch > > > source gives syntax errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2568) HIVE-2246 upgrade script needs to drop foreign key in COLUMNS_OLD
[ https://issues.apache.org/jira/browse/HIVE-2568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-2568: -- @Ning: Can you please backport this 0.8.0? Thanks. > HIVE-2246 upgrade script needs to drop foreign key in COLUMNS_OLD > - > > Key: HIVE-2568 > URL: https://issues.apache.org/jira/browse/HIVE-2568 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.8.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.8.0, 0.9.0 > > Attachments: D369.1.patch > > > One more bug in the MySQL metastore upgrade script: the foreign key in > COLUMNS needs to be dropped, otherwise drop_partition will fail because the > SDS row cannot be deleted due to the foreign key constraint. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2568) HIVE-2246 upgrade script needs to drop foreign key in COLUMNS_OLD
[ https://issues.apache.org/jira/browse/HIVE-2568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2568: - Priority: Blocker (was: Major) > HIVE-2246 upgrade script needs to drop foreign key in COLUMNS_OLD > - > > Key: HIVE-2568 > URL: https://issues.apache.org/jira/browse/HIVE-2568 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.8.0 >Reporter: Ning Zhang >Assignee: Ning Zhang >Priority: Blocker > Fix For: 0.8.0, 0.9.0 > > Attachments: D369.1.patch > > > One more bug in the MySQL metastore upgrade script: the foreign key in > COLUMNS needs to be dropped, otherwise drop_partition will fail because the > SDS row cannot be deleted due to the foreign key constraint. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2568) HIVE-2246 upgrade script needs to drop foreign key in COLUMNS_OLD
[ https://issues.apache.org/jira/browse/HIVE-2568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2568: - Component/s: Metastore > HIVE-2246 upgrade script needs to drop foreign key in COLUMNS_OLD > - > > Key: HIVE-2568 > URL: https://issues.apache.org/jira/browse/HIVE-2568 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.8.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.8.0, 0.9.0 > > Attachments: D369.1.patch > > > One more bug in the MySQL metastore upgrade script: the foreign key in > COLUMNS needs to be dropped, otherwise drop_partition will fail because the > SDS row cannot be deleted due to the foreign key constraint. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2572) HIVE-2246 upgrade script changed the COLUMNS_V2.COMMENT length
[ https://issues.apache.org/jira/browse/HIVE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2572: - Component/s: Metastore Priority: Blocker (was: Major) > HIVE-2246 upgrade script changed the COLUMNS_V2.COMMENT length > -- > > Key: HIVE-2572 > URL: https://issues.apache.org/jira/browse/HIVE-2572 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.8.0 >Reporter: Ning Zhang >Assignee: Ning Zhang >Priority: Blocker > Fix For: 0.8.0, 0.9.0 > > Attachments: D381.1.patch > > > This changes from varchar(4000) to varchar(128) is backward incompatible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2572) HIVE-2246 upgrade script changed the COLUMNS_V2.COMMENT length
[ https://issues.apache.org/jira/browse/HIVE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-2572: -- @Ning: Can you please backport this to 0.8.0? Thanks. > HIVE-2246 upgrade script changed the COLUMNS_V2.COMMENT length > -- > > Key: HIVE-2572 > URL: https://issues.apache.org/jira/browse/HIVE-2572 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.8.0 >Reporter: Ning Zhang >Assignee: Ning Zhang >Priority: Blocker > Fix For: 0.8.0, 0.9.0 > > Attachments: D381.1.patch > > > This changes from varchar(4000) to varchar(128) is backward incompatible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2574) ivy offline mode broken by changingPattern and checkmodified attributes
[ https://issues.apache.org/jira/browse/HIVE-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2574: -- Attachment: HIVE-2574.D435.2.patch jsichi updated the revision "HIVE-2574 [jira] ivy offline mode broken by changingPattern and checkmodified attributes". Reviewers: JIRA, mareksapotafb Tentative change to see if we can avoid ivy concurrency problems. REVISION DETAIL https://reviews.facebook.net/D435 AFFECTED FILES build.xml ivy/ivysettings.xml build-common.xml build-offline.xml > ivy offline mode broken by changingPattern and checkmodified attributes > --- > > Key: HIVE-2574 > URL: https://issues.apache.org/jira/browse/HIVE-2574 > Project: Hive > Issue Type: Bug >Reporter: John Sichi >Assignee: John Sichi > Attachments: HIVE-2574.D435.1.patch, HIVE-2574.D435.2.patch > > > As described here: > http://www.mail-archive.com/ivy-user@ant.apache.org/msg03534.html > This wasn't the case formerly (maybe the upgrade to ivy 1.2?) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2579) Support HTTP URIs for ADD JAR
Support HTTP URIs for ADD JAR - Key: HIVE-2579 URL: https://issues.apache.org/jira/browse/HIVE-2579 Project: Hive Issue Type: New Feature Reporter: David Phillips Priority: Trivial It would be very useful for ADD JAR to support HTTP/HTTPS. For example, this allows one to deploy a UDF jar to a Maven repository and then access it directly from the repository (which provides files via HTTP). I realize this could be handled by adding a Hadoop FileSystem for HTTP, but that file system wouldn't be generally useful as HTTP doesn't support listing or modifying files (without something like WebDAV). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2508) Join on union type fails
[
https://issues.apache.org/jira/browse/HIVE-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-2508:
-
Component/s: Query Processor
> Join on union type fails
>
>
> Key: HIVE-2508
> URL: https://issues.apache.org/jira/browse/HIVE-2508
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Ashutosh Chauhan
> Labels: uniontype
>
> {code}
> hive> CREATE TABLE DEST1(key UNIONTYPE, value BIGINT) STORED
> AS TEXTFILE;
> OK
> Time taken: 0.076 seconds
> hive> CREATE TABLE DEST2(key UNIONTYPE, value BIGINT) STORED
> AS TEXTFILE;
> OK
> Time taken: 0.034 seconds
> hive> SELECT * FROM DEST1 JOIN DEST2 on (DEST1.key = DEST2.key);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2573: Attachment: HIVE-2573.2.patch.txt added test query & result > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-2573: create per-session function registry
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2831/ --- Review request for hive and Carl Steinbach. Summary --- Extension from HIVE-2503 for function registry. This addresses bug HIVE-2573. https://issues.apache.org/jira/browse/HIVE-2573 Diffs - ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 35d124b ql/src/test/queries/clientpositive/session_function.q PRE-CREATION ql/src/test/results/clientpositive/session_function.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 4d4fce2 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d60ba09 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateFunctionDesc.java 051095a ql/src/java/org/apache/hadoop/hive/ql/plan/DropFunctionDesc.java 8a78f5b ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 07a4832 ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java f2b7018 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 70b6f57 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 197bc77 Diff: https://reviews.apache.org/r/2831/diff Testing --- Thanks, Navis
[jira] [Commented] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150132#comment-13150132 ] jirapos...@reviews.apache.org commented on HIVE-2573: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2831/ --- Review request for hive and Carl Steinbach. Summary --- Extension from HIVE-2503 for function registry. This addresses bug HIVE-2573. https://issues.apache.org/jira/browse/HIVE-2573 Diffs - ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 35d124b ql/src/test/queries/clientpositive/session_function.q PRE-CREATION ql/src/test/results/clientpositive/session_function.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 4d4fce2 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d60ba09 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateFunctionDesc.java 051095a ql/src/java/org/apache/hadoop/hive/ql/plan/DropFunctionDesc.java 8a78f5b ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 07a4832 ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java f2b7018 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 70b6f57 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 197bc77 Diff: https://reviews.apache.org/r/2831/diff Testing --- Thanks, Navis > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2574) ivy offline mode broken by changingPattern and checkmodified attributes
[ https://issues.apache.org/jira/browse/HIVE-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2574: -- Attachment: HIVE-2574.D435.3.patch jsichi updated the revision "HIVE-2574 [jira] ivy offline mode broken by changingPattern and checkmodified attributes". Reviewers: JIRA, mareksapotafb Turns out we only need to set resolutionCacheDir. REVISION DETAIL https://reviews.facebook.net/D435 AFFECTED FILES build.xml ivy/ivysettings.xml build-common.xml build-offline.xml > ivy offline mode broken by changingPattern and checkmodified attributes > --- > > Key: HIVE-2574 > URL: https://issues.apache.org/jira/browse/HIVE-2574 > Project: Hive > Issue Type: Bug >Reporter: John Sichi >Assignee: John Sichi > Attachments: HIVE-2574.D435.1.patch, HIVE-2574.D435.2.patch, > HIVE-2574.D435.3.patch > > > As described here: > http://www.mail-archive.com/ivy-user@ant.apache.org/msg03534.html > This wasn't the case formerly (maybe the upgrade to ivy 1.2?) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2580) Hive build fails with error "java.io.IOException: Not in GZIP format"
Hive build fails with error "java.io.IOException: Not in GZIP format" - Key: HIVE-2580 URL: https://issues.apache.org/jira/browse/HIVE-2580 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0 Reporter: Carl Steinbach Assignee: Carl Steinbach Priority: Blocker Fix For: 0.8.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2580) Hive build fails with error "java.io.IOException: Not in GZIP format"
[
https://issues.apache.org/jira/browse/HIVE-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150174#comment-13150174
]
Carl Steinbach commented on HIVE-2580:
--
HIVE-2391 upgraded the Ivy dependency from 2.1.0 to 2.2.0. It looks like 2.2.0
contains a bug (IVY-1320) that causes the build to fail if the user has a
a fresh .ivy2 directory, e.g:
{noformat}
BUILD FAILED
/home/carl/Work/repos/hive2/build.xml:297: The following error occurred while
executing this line:
/home/carl/Work/repos/hive2/build.xml:154: The following error occurred while
executing this line:
/home/carl/Work/repos/hive2/shims/build.xml:65: The following error occurred
while executing this line:
/home/carl/Work/repos/hive2/build-common.xml:132: Error while expanding
/home/carl/Work/repos/hive2/build/hadoopcore/hadoop-0.20.1.tar.gz
java.io.IOException: Not in GZIP format
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:143)
at java.util.zip.GZIPInputStream.(GZIPInputStream.java:58)
at java.util.zip.GZIPInputStream.(GZIPInputStream.java:67)
at
org.apache.tools.ant.taskdefs.Untar$UntarCompressionMethod.decompress(Untar.java:235)
at org.apache.tools.ant.taskdefs.Untar.expandStream(Untar.java:158)
at org.apache.tools.ant.taskdefs.Untar.expandFile(Untar.java:114)
at org.apache.tools.ant.taskdefs.Expand.execute(Expand.java:132)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at
org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at
org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:302)
at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:221)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:68)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at
org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:398)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(Dispatc
Re: Review Request: HIVE-2573: create per-session function registry
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2831/ --- (Updated 2011-11-15 02:14:32.702033) Review request for hive and Carl Steinbach. Summary --- Extension from HIVE-2503 for function registry. This addresses bug HIVE-2573. https://issues.apache.org/jira/browse/HIVE-2573 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java f2b7018 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 4d4fce2 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateFunctionDesc.java 051095a ql/src/java/org/apache/hadoop/hive/ql/plan/DropFunctionDesc.java 8a78f5b ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 07a4832 ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 35d124b ql/src/test/queries/clientpositive/session_function.q PRE-CREATION ql/src/test/results/clientpositive/session_function.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 70b6f57 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 197bc77 Diff: https://reviews.apache.org/r/2831/diff Testing --- Thanks, Navis
[jira] [Commented] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150183#comment-13150183 ] jirapos...@reviews.apache.org commented on HIVE-2573: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2831/ --- (Updated 2011-11-15 02:14:32.702033) Review request for hive and Carl Steinbach. Summary --- Extension from HIVE-2503 for function registry. This addresses bug HIVE-2573. https://issues.apache.org/jira/browse/HIVE-2573 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java f2b7018 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 4d4fce2 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateFunctionDesc.java 051095a ql/src/java/org/apache/hadoop/hive/ql/plan/DropFunctionDesc.java 8a78f5b ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 07a4832 ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 35d124b ql/src/test/queries/clientpositive/session_function.q PRE-CREATION ql/src/test/results/clientpositive/session_function.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 70b6f57 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 197bc77 Diff: https://reviews.apache.org/r/2831/diff Testing --- Thanks, Navis > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1085 - Failure
Changes for Build #1085 [namit] HIVE-2196 Ensure HiveConf includes all properties defined in hive-default.xml (Chinna Rao Lalam via namit) 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:9332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1085) Status: Failure Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1085/ to view the results.
[jira] [Commented] (HIVE-2196) Ensure HiveConf includes all properties defined in hive-default.xml
[ https://issues.apache.org/jira/browse/HIVE-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150187#comment-13150187 ] Hudson commented on HIVE-2196: -- Integrated in Hive-trunk-h0.21 #1085 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1085/]) HIVE-2196 Ensure HiveConf includes all properties defined in hive-default.xml (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201872 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > Ensure HiveConf includes all properties defined in hive-default.xml > --- > > Key: HIVE-2196 > URL: https://issues.apache.org/jira/browse/HIVE-2196 > Project: Hive > Issue Type: Bug > Components: Configuration >Affects Versions: 0.8.0 >Reporter: Carl Steinbach >Assignee: Chinna Rao Lalam > Attachments: HIVE-2196.1.patch, HIVE-2196.2.patch, HIVE-2196.3.patch, > HIVE-2196.4.patch, HIVE-2196.5.patch, HIVE-2196.6.patch, HIVE-2196.build.log, > HIVE-2196.patch > > > There are a bunch of properties that are defined in hive-default.xml but not > in HiveConf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2580) Hive build fails with error "java.io.IOException: Not in GZIP format"
[ https://issues.apache.org/jira/browse/HIVE-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2580: - Attachment: HIVE-2580.1.patch.txt > Hive build fails with error "java.io.IOException: Not in GZIP format" > - > > Key: HIVE-2580 > URL: https://issues.apache.org/jira/browse/HIVE-2580 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.8.0 >Reporter: Carl Steinbach >Assignee: Carl Steinbach >Priority: Blocker > Fix For: 0.8.0 > > Attachments: HIVE-2580.1.patch.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2580) Hive build fails with error "java.io.IOException: Not in GZIP format"
[ https://issues.apache.org/jira/browse/HIVE-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2580: - Status: Patch Available (was: Open) Attached patch downgrades the Ivy dependency to version 2.1.0. Verified that 'ant clean package maven-build maven-publish' works. Running full tests now. If someone can apply this to trunk I will handle backporting it to branch-0.8. Thanks! > Hive build fails with error "java.io.IOException: Not in GZIP format" > - > > Key: HIVE-2580 > URL: https://issues.apache.org/jira/browse/HIVE-2580 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.8.0 >Reporter: Carl Steinbach >Assignee: Carl Steinbach >Priority: Blocker > Fix For: 0.8.0 > > Attachments: HIVE-2580.1.patch.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-80) Add testcases for concurrent query execution
[ https://issues.apache.org/jira/browse/HIVE-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150193#comment-13150193 ] Konstantin Boudnik commented on HIVE-80: Perhaps my question is bigger than this issue or shall be asked somewhere else, but looking at Hive code I couldn't help noticing that it is essentially build as a singleton i.e. only a single instance of Hive object is allowed to exist. What was/is a design motivation behind of this? Why Hive can't be instantiated at will by the client so different instances can be independently for query analysis, job submissions, etc.? This will make Hive much more flexible and extendable from Hive client applications perspective, won't it? > Add testcases for concurrent query execution > > > Key: HIVE-80 > URL: https://issues.apache.org/jira/browse/HIVE-80 > Project: Hive > Issue Type: Test > Components: Query Processor, Server Infrastructure >Reporter: Raghotham Murthy >Assignee: Carl Steinbach >Priority: Critical > Labels: concurrency > Attachments: hive_input_format_race-2.patch > > > Can use one driver object per query. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2580) Hive build fails with error "java.io.IOException: Not in GZIP format"
[ https://issues.apache.org/jira/browse/HIVE-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150198#comment-13150198 ] John Sichi commented on HIVE-2580: -- +1, will commit when tests pass. > Hive build fails with error "java.io.IOException: Not in GZIP format" > - > > Key: HIVE-2580 > URL: https://issues.apache.org/jira/browse/HIVE-2580 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.8.0 >Reporter: Carl Steinbach >Assignee: Carl Steinbach >Priority: Blocker > Fix For: 0.8.0 > > Attachments: HIVE-2580.1.patch.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150214#comment-13150214 ] Carl Steinbach commented on HIVE-2573: -- bq. I've thought keyword 'TEMPORARY' could mean the 'session' thing but it seemed to be not safe. What issues did you run into? I think the original intent of the 'TEMPORARY' keyword was to signal that the function's lifecycle is tied to the session. Adding a new SESSION keyword seems to imply that non-SESSION TEMPORARY functions should be globally visible, which doesn't seem like a good idea. If we currently have tests that rely on TEMPORARY functions being visible to other sessions, then maybe we should instead support 'CREATE [TEMPORARY] FUNCTION ...', mapping 'CREATE FUNCTION ...' to the current behavior, and 'CREATE TEMPORARY FUNCTION ...' to the new behavior added in this patch. > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2467) HA Support for Metastore Server
[ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150243#comment-13150243 ] Andreas Neumann commented on HIVE-2467: --- If Hadoop .23 relies on ZooKeeper, then I would hope that ZK is already baked in... otherwise I would tend to think, if ZK is tolerably secure for Hadoop, then it should be tolerable for Hive/HCatalog. After all, afaik both Hive and HCatalog piggy back on HDFS for security. But I like the idea of having different implementations for the TokenStore, even though my preferred deployment would be ZooKeeper-based. > HA Support for Metastore Server > > > Key: HIVE-2467 > URL: https://issues.apache.org/jira/browse/HIVE-2467 > Project: Hive > Issue Type: Improvement > Components: Metastore, Security, Server Infrastructure >Affects Versions: 0.8.0, 0.9.0 >Reporter: Thomas Weise >Assignee: Thomas Weise > Fix For: 0.9.0 > > Attachments: HIVE-2467.patch > > > We require HA deployment for metastore server for HCatalog: > * Multiple server instances run behind VIP > * Database provides HA > Metastore server instances will need to be able to share any state required > for VIP outside RDBMS. As of Hive 0.8 affected conversational state that > needs to support VIP/HA setup is limited to current delegation tokens. Is > this correct? > We are planning to use ZooKeeper to share current delegation tokens and > master keys between nodes of the VIP. ZK is already (optionally) used by Hive > for concurrency control. Access to ZK would be limited on the network level > or in the future, when ZooKeeper supports security, through Kerberos, similar > to NN access. > Currently Hive taps into Hadoop core security delegation token support > through extension of > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager > A solution could amend the Hive specific extension to support: > * Pluggable delegation token and master key store (ZooKeeper as alternative > for in-memory AbstractDelegationTokenSecretManager) > * Delegation token retrieval from token store when not found in memory > (wrap/extend retrievePassword(...)) > * Cancellation of token in token store > * Purging of expired tokens from token store > http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150244#comment-13150244 ] Navis commented on HIVE-2573: - I agree and can say that first modification was exactly same as you mentioned. But I was not certain that someone might be relying on the current globalness of temporary function (furthermore, keyword 'TEMPORARY' was mandatory). I'll revise this patch shortly. > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2489) Add configuration properties for HiveServer and HiveMetaStore port numbers
[ https://issues.apache.org/jira/browse/HIVE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] iryoung jeong updated HIVE-2489: Attachment: HIVE-2489.patch After some research about the issue, it may need some design decisions. There're 3 ways to set port now - using default port, from cli(-p or old style), HIVE_PORT/METASTORE_PORT system env. And after this issue is fixed, there'll be another way to configure port. So, it's necessary cleaning up these. Previous setting port precedence looks like this "default port < system env < CLI option" (if higher one is set, then use it) and, now it'll be "default port < system env < HiveConf < CLI option" IMO, all other settings should be finally collected into a HiveConf object. So I implemented this way for HiveMetaStore. But in HiveServer, some configurations are only available through CLI options, so I chose applying port number from HiveConf to previous CLI object.(which is required minimum code change, and I thought moving other settings to HiveConf is out of scope of this issue) I'll attached a initial patch. > Add configuration properties for HiveServer and HiveMetaStore port numbers > -- > > Key: HIVE-2489 > URL: https://issues.apache.org/jira/browse/HIVE-2489 > Project: Hive > Issue Type: New Feature > Components: Configuration, Metastore >Reporter: Carl Steinbach > Attachments: HIVE-2489.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2573: Attachment: HIVE-2573.3.patch.txt > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt, > HIVE-2573.3.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150271#comment-13150271 ] jirapos...@reviews.apache.org commented on HIVE-2573: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2831/ --- (Updated 2011-11-15 07:06:04.002057) Review request for hive and Carl Steinbach. Changes --- Changed to use KW_TEMPORARY instead of 'SESSION'. Some security checks would be needed for 'CREATE FUNCTION' which would add function to main function registry. Summary --- Extension from HIVE-2503 for function registry. This addresses bug HIVE-2573. https://issues.apache.org/jira/browse/HIVE-2573 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 4d4fce2 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateFunctionDesc.java 051095a ql/src/java/org/apache/hadoop/hive/ql/plan/DropFunctionDesc.java 8a78f5b ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 07a4832 ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 35d124b ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 197bc77 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 70b6f57 ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java f2b7018 Diff: https://reviews.apache.org/r/2831/diff Testing --- Thanks, Navis > Create per-session function registry > - > > Key: HIVE-2573 > URL: https://issues.apache.org/jira/browse/HIVE-2573 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: Navis >Priority: Minor > Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt, > HIVE-2573.3.patch.txt > > > Currently the function registry is shared resource and could be overrided by > other users when using HiveServer. If per-session function registry is > provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2573: create per-session function registry
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2831/ --- (Updated 2011-11-15 07:06:04.002057) Review request for hive and Carl Steinbach. Changes --- Changed to use KW_TEMPORARY instead of 'SESSION'. Some security checks would be needed for 'CREATE FUNCTION' which would add function to main function registry. Summary --- Extension from HIVE-2503 for function registry. This addresses bug HIVE-2573. https://issues.apache.org/jira/browse/HIVE-2573 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 4d4fce2 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateFunctionDesc.java 051095a ql/src/java/org/apache/hadoop/hive/ql/plan/DropFunctionDesc.java 8a78f5b ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 07a4832 ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 35d124b ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 197bc77 ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 70b6f57 ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java f2b7018 Diff: https://reviews.apache.org/r/2831/diff Testing --- Thanks, Navis
