[jira] [Commented] (HIVE-2894) RCFile Reader doesn't provide access to Metadata
[ https://issues.apache.org/jira/browse/HIVE-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237464#comment-13237464 ] Hudson commented on HIVE-2894: -- Integrated in Hive-trunk-h0.21 #1330 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1330/]) HIVE-2894 [jira] RCFile Reader doesn't provide access to Metadata (Owen O'Malley via Ashutosh Chauhan) Summary: hive-2894 Add an accessor for RCFile's metadata. Currently the RCFile writer can add metadata to an RCFile, but the reader doesn't provide an accessor. I'd like to add one. Test Plan: I added a call to test that the metadata that was passed in was available from the reader. Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc Differential Revision: https://reviews.facebook.net/D2421 (Revision 1304693) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1304693 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java RCFile Reader doesn't provide access to Metadata Key: HIVE-2894 URL: https://issues.apache.org/jira/browse/HIVE-2894 Project: Hive Issue Type: New Feature Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.9.0 Attachments: HIVE-2894.D2421.1.patch Currently the RCFile writer can add metadata to an RCFile, but the reader doesn't provide an accessor. I'd like to add one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1330 - Fixed
Changes for Build #1326 [hashutosh] HIVE-2819 : Closed range scans on hbase keys : Reviewed by Carl Steinbach (hashutosh) Changes for Build #1327 [hashutosh] HIVE-2577: Expose the HiveConf in HiveConnection API (Nicolas Lalevee via Ashutosh Chauhan) Changes for Build #1328 [hashutosh] HIVE-2773: HiveStorageHandler.configureTableJobProperites() should let the handler know wether it is configuration for input or output (Francis Liu via Ashutosh Chauhan) Changes for Build #1329 Changes for Build #1330 [hashutosh] HIVE-2894 [jira] RCFile Reader doesn't provide access to Metadata (Owen O'Malley via Ashutosh Chauhan) Summary: hive-2894 Add an accessor for RCFile's metadata. Currently the RCFile writer can add metadata to an RCFile, but the reader doesn't provide an accessor. I'd like to add one. Test Plan: I added a call to test that the metadata that was passed in was available from the reader. Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc Differential Revision: https://reviews.facebook.net/D2421 All tests passed The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1330) Status: Fixed Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1330/ to view the results.
[jira] [Updated] (HIVE-2822) Add JSON output to the hive ddl commands
[
https://issues.apache.org/jira/browse/HIVE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2822:
--
Attachment: HIVE-2822.D2475.1.patch
ctdean requested code review of HIVE-2822 [jira] Add JSON output to the hive
ddl commands.
Reviewers: JIRA
JSON output for DDL commands
By setting the variable hive.format=json then ddl commands will
produce json output suitable for client side parsing. For example, to
list all the databases one might get:
{
databases: [
default
]
}
The goal is to have an option to produce JSON output of the DDL commands that
is easily machine parseable.
For example, desc my_table currently gives
idbigint
user string
and we want to allow a json output:
{
columns: [
{name: id, type: bigint},
{name: user, type: string}
]
}
TEST PLAN
Run the provided unit tests
REVISION DETAIL
https://reviews.facebook.net/D2475
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
ql/src/java/org/apache/hadoop/hive/ql/metadata/JsonMetaDataFormatter.java
ql/src/java/org/apache/hadoop/hive/ql/metadata/MapBuilder.java
ql/src/java/org/apache/hadoop/hive/ql/metadata/MetaDataFormatter.java
ql/src/java/org/apache/hadoop/hive/ql/metadata/TextMetaDataFormatter.java
ql/src/test/queries/clientpositive/describe_database_json.q
ql/src/test/queries/clientpositive/describe_table_json.q
ql/src/test/queries/clientpositive/misc_json.q
ql/src/test/queries/clientpositive/partitions_json.q
ql/src/test/results/clientpositive/describe_database_json.q.out
ql/src/test/results/clientpositive/describe_table_json.q.out
ql/src/test/results/clientpositive/misc_json.q.out
ql/src/test/results/clientpositive/partitions_json.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/5553/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
Add JSON output to the hive ddl commands
Key: HIVE-2822
URL: https://issues.apache.org/jira/browse/HIVE-2822
Project: Hive
Issue Type: Improvement
Reporter: Chris Dean
Attachments: HIVE-2822.03-branch0-8.patch, HIVE-2822.03.patch,
HIVE-2822.03b.patch, HIVE-2822.04-branch-08.patch,
HIVE-2822.05-branch0-8-1.patch, HIVE-2822.05-branch0-8.patch,
HIVE-2822.05.patch, HIVE-2822.D2475.1.patch, hive-json-01-branch0-8.patch,
hive-json-01.patch, hive-json-02-branch0-8.patch, hive-json-02.patch
The goal is to have an option to produce JSON output of the DDL commands that
is easily machine parseable.
For example, desc my_table currently gives
{noformat}
idbigint
user string
{noformat}
and we want to allow a json output:
{noformat}
{
columns: [
{name: id, type: bigint},
{name: user, type: string}
]
}
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2518) pull junit jar from maven repos via ivy
[ https://issues.apache.org/jira/browse/HIVE-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237752#comment-13237752 ] Ashutosh Chauhan commented on HIVE-2518: No worries : ) Thanks for the update. pull junit jar from maven repos via ivy --- Key: HIVE-2518 URL: https://issues.apache.org/jira/browse/HIVE-2518 Project: Hive Issue Type: Improvement Reporter: He Yongqiang Assignee: Kevin Wilfong see https://issues.apache.org/jira/browse/HIVE-2505 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.
[ https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2797: Attachment: HIVE-2797.7.patch Make the IP address of a Thrift client available to HMSHandler. --- Key: HIVE-2797 URL: https://issues.apache.org/jira/browse/HIVE-2797 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2797.7.patch, HIVE-2797.D1701.1.patch, HIVE-2797.D1701.2.patch, HIVE-2797.D1701.3.patch, HIVE-2797.D1701.4.patch, HIVE-2797.D1701.5.patch, HIVE-2797.D1701.6.patch Currently, in unsecured mode, metastore Thrift calls are, from the HMSHandler's point of view, anonymous. If we expose the IP address of the Thrift client to the HMSHandler from the Processor, this will help to give some context, in particular for audit logging, of where the call is coming from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.
[ https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237754#comment-13237754 ] Kevin Wilfong commented on HIVE-2797: - Looks like something my editor must have done automatically, something about adding or removing a new line at the end of files. Anyway, I removed those lines from the patch manually (I hadn't intended to change that line anyway). HIVE-2797.7.patch contains this, and I tried applying it to a fresh checkout and did not get that error. Make the IP address of a Thrift client available to HMSHandler. --- Key: HIVE-2797 URL: https://issues.apache.org/jira/browse/HIVE-2797 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2797.7.patch, HIVE-2797.D1701.1.patch, HIVE-2797.D1701.2.patch, HIVE-2797.D1701.3.patch, HIVE-2797.D1701.4.patch, HIVE-2797.D1701.5.patch, HIVE-2797.D1701.6.patch Currently, in unsecured mode, metastore Thrift calls are, from the HMSHandler's point of view, anonymous. If we expose the IP address of the Thrift client to the HMSHandler from the Processor, this will help to give some context, in particular for audit logging, of where the call is coming from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2875) Renaming partition changes partition location prefix
[ https://issues.apache.org/jira/browse/HIVE-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2875: Status: Patch Available (was: Open) Renaming partition changes partition location prefix Key: HIVE-2875 URL: https://issues.apache.org/jira/browse/HIVE-2875 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2875.D2349.1.patch, HIVE-2875.D2349.2.patch Renaming a partition changes the location of the partition to the default location of the table, followed by the partition specification. It should just change the partition specification of the path. If the path does not end with the old partition specification, we should probably throw an exception because renaming a partition should not change the path so dramatically, and not changing the path to reflect the new partition name could leave the partition in a very confusing state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2875) Renaming partition changes partition location prefix
[ https://issues.apache.org/jira/browse/HIVE-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2875: -- Attachment: HIVE-2875.D2349.2.patch kevinwilfong updated the revision HIVE-2875 [jira] Renaming partition changes partition location prefix. Reviewers: JIRA, njain I modified my approach. Now the partition's new location consists of the scheme and authority of the original location, and the path of the default location using the partition's new name. This is similar to what is done for partition's locations when a table is renamed. I ran the tests and they now pass. REVISION DETAIL https://reviews.facebook.net/D2349 AFFECTED FILES metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java ql/src/test/results/clientpositive/rename_partition_location.q.out ql/src/test/org/apache/hadoop/hive/ql/hooks/VerifyPartitionIsNotSubdirectoryOfTableHook.java ql/src/test/queries/clientpositive/rename_partition_location.q Renaming partition changes partition location prefix Key: HIVE-2875 URL: https://issues.apache.org/jira/browse/HIVE-2875 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2875.D2349.1.patch, HIVE-2875.D2349.2.patch Renaming a partition changes the location of the partition to the default location of the table, followed by the partition specification. It should just change the partition specification of the path. If the path does not end with the old partition specification, we should probably throw an exception because renaming a partition should not change the path so dramatically, and not changing the path to reflect the new partition name could leave the partition in a very confusing state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.
[ https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2797: Status: Patch Available (was: Open) Make the IP address of a Thrift client available to HMSHandler. --- Key: HIVE-2797 URL: https://issues.apache.org/jira/browse/HIVE-2797 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2797.7.patch, HIVE-2797.D1701.1.patch, HIVE-2797.D1701.2.patch, HIVE-2797.D1701.3.patch, HIVE-2797.D1701.4.patch, HIVE-2797.D1701.5.patch, HIVE-2797.D1701.6.patch Currently, in unsecured mode, metastore Thrift calls are, from the HMSHandler's point of view, anonymous. If we expose the IP address of the Thrift client to the HMSHandler from the Processor, this will help to give some context, in particular for audit logging, of where the call is coming from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2902) undefined property exists in eclipse-templates/.classpath
undefined property exists in eclipse-templates/.classpath
-
Key: HIVE-2902
URL: https://issues.apache.org/jira/browse/HIVE-2902
Project: Hive
Issue Type: Bug
Components: Build Infrastructure
Reporter: tamtam180
Priority: Minor
@hbase-test.version@ was removed from ivy/libraries.properties in HIVE-2748,
but the property still exists in eclipse-templates/.classpath.
{code}
classpathentry kind=lib path=build/ivy/lib/default/hbase-0.92.0.jar/
classpathentry kind=lib
path=build/ivy/lib/default/hbase-@hbase-test.vers...@-tests.jar/
{code}
It should be changed to @hbase.version@
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2903) Numeric binary type keys are not compared properly
Numeric binary type keys are not compared properly
--
Key: HIVE-2903
URL: https://issues.apache.org/jira/browse/HIVE-2903
Project: Hive
Issue Type: Bug
Components: HBase Handler
Reporter: Navis
Assignee: Navis
In current binary format for numbers, minus values are always greater than plus
values, for example.
{code}
System.our.println(Bytes.compareTo(Bytes.toBytes(-100), Bytes.toBytes(100)));
// 255
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2903) Numeric binary type keys are not compared properly
[
https://issues.apache.org/jira/browse/HIVE-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2903:
--
Attachment: HIVE-2903.D2481.1.patch
navis requested code review of HIVE-2903 [jira] Numeric binary type keys are
not compared properly.
Reviewers: JIRA
DPAL-1007 Numeric binary type keys are not compared properly
In current binary format for numbers, minus values are always greater than
plus values, for example.
System.our.println(Bytes.compareTo(Bytes.toBytes(-100), Bytes.toBytes(100)));
// 255
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2481
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
hbase-handler/src/test/queries/external_table_ppd.q
hbase-handler/src/test/results/external_table_ppd.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/5565/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
Numeric binary type keys are not compared properly
--
Key: HIVE-2903
URL: https://issues.apache.org/jira/browse/HIVE-2903
Project: Hive
Issue Type: Bug
Components: HBase Handler
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2903.D2481.1.patch
In current binary format for numbers, minus values are always greater than
plus values, for example.
{code}
System.our.println(Bytes.compareTo(Bytes.toBytes(-100), Bytes.toBytes(100)));
// 255
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2897) Remove filter operator of which all the predicates have pushed down to storage handler
[ https://issues.apache.org/jira/browse/HIVE-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237776#comment-13237776 ] Navis commented on HIVE-2897: - After removing filter operator additional rows are found in result, which was caused by HIVE-2903. It should be fixed first. Remove filter operator of which all the predicates have pushed down to storage handler -- Key: HIVE-2897 URL: https://issues.apache.org/jira/browse/HIVE-2897 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-2897.D2427.1.patch If all the predicates have pushed down to StorageHandler in TS, it should be removed (or make empty). Might it be intentional? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
