gt;
> https://github.com/apache/hive
> https://github.com/apache/hive-iceberg
>
> Thanks.
>
> On Wed, Mar 3, 2021 at 12:30 PM Ryan Blue
> wrote:
>
>> I think that this direction sounds reasonable. It makes sense to start
>> building the integration in Hive because it wi
an decide our next steps
> based on the actual landscape, and in the meantime we can port the changes
> between the 2 repo which does not require the new APIs.
>
> I would like to hear both teams opinion of the proposed solution.
>
> Thanks,
> Peter
>
--
Ryan Blue
Software Engineer
Netflix
rter.java
(line 214)
<https://reviews.apache.org/r/41821/#comment176985>
This is a great test for round trip, but I think we should also have a test
with a set of corrupt values from a file written with 5.5 in
America/Los_Angeles (since that's convenient).
- Ryan Blue
On Jan.
Ryan Blue created HIVE-12197:
Summary: Parquet timestamps differ between Hive and Impala
Key: HIVE-12197
URL: https://issues.apache.org/jira/browse/HIVE-12197
Project: Hive
Issue Type: Sub-task
Ryan Blue created HIVE-12196:
Summary: NPE when converting bad timestamp value
Key: HIVE-12196
URL: https://issues.apache.org/jira/browse/HIVE-12196
Project: Hive
Issue Type: Sub-task
Ryan Blue created HIVE-12195:
Summary: Unknown zones should cause an error instead of silently
failing
Key: HIVE-12195
URL: https://issues.apache.org/jira/browse/HIVE-12195
Project: Hive
Issue
Ryan Blue created HIVE-12194:
Summary: Incorrect result when using from_utc_timestamp with the
local time zone.
Key: HIVE-12194
URL: https://issues.apache.org/jira/browse/HIVE-12194
Project: Hive
Ryan Blue created HIVE-12193:
Summary: Add a consistent unix_timestamp function.
Key: HIVE-12193
URL: https://issues.apache.org/jira/browse/HIVE-12193
Project: Hive
Issue Type: Sub-task
Ryan Blue created HIVE-12192:
Summary: Hive should carry out timestamp computations in UTC
Key: HIVE-12192
URL: https://issues.apache.org/jira/browse/HIVE-12192
Project: Hive
Issue Type: Sub
Ryan Blue created HIVE-12191:
Summary: Hive timestamp problems
Key: HIVE-12191
URL: https://issues.apache.org/jira/browse/HIVE-12191
Project: Hive
Issue Type: Bug
Components: Hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35950/#review89812
---
Ship it!
Ship It!
- Ryan Blue
On June 27, 2015, 5:29 p.m
he/hadoop/hive/ql/io/parquet/TestParquetToHiveSchemaConverter.java
<https://reviews.apache.org/r/28372/#comment132401>
I think the maps and lists should be constructed using the current best
practice, ConversionPatterns, instead of by hand. That way we're testing what
is actually going to be in data files.
t/read/DataWritableReadSupport.java
<https://reviews.apache.org/r/32499/#comment126309>
Why `new Type[0]` instead of `new Type[projectedFields.size()]`?
- Ryan Blue
On March 26, 2015, 1:51 p.m., Sergio Pena wrote:
>
> ---
> This is an auto
te to rename the
constant's value, but renaming the variable might help readability.
ql/src/test/queries/clientpositive/parquet_table_with_subschema.q
<https://reviews.apache.org/r/32499/#comment126261>
Could you reverse the order of zip and Street? I want t
tInspector.java
<https://reviews.apache.org/r/31800/#comment122776>
Should this be named `arrObjects`?
- Ryan Blue
On March 6, 2015, 9:18 a.m., Sergio Pena wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30717/#review73933
---
Ship it!
Ship It!
- Ryan Blue
On Feb. 5, 2015, 11:51 p.m., Dong
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30281/#review72398
---
Ship it!
Ship It!
- Ryan Blue
On Feb. 11, 2015, 3:19 p.m
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30750/#review72067
---
Ship it!
Ship It!
- Ryan Blue
On Feb. 11, 2015, 3:51 p.m
is would be a good place, so I
think this should actually be "array" => "list" and "element" for elements.
That way we're testing the new structure exactly.
- Ryan Blue
On Feb. 11, 2015, 3:19 p.m., Sergio Pena wrote:
>
> ---
onConverter.java
<https://reviews.apache.org/r/30750/#comment117743>
Nit: relocating import statements
- Ryan Blue
On Feb. 10, 2015, 1:25 p.m., Sergio Pena wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit
et/TestDataWritableWriter.java
<https://reviews.apache.org/r/30281/#comment117740>
These name changes should be reverted, they previous names are required by
PARQUET-113.
- Ryan Blue
On Jan. 29, 2015, 9:12 a.m., Sergio Pena wrote:
>
> -
epetition, INT32).as(DATE).named(name);
```
- Ryan Blue
On Feb. 5, 2015, 11:51 p.m., Dong Chen wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https:
[
https://issues.apache.org/jira/browse/HIVE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312604#comment-14312604
]
Ryan Blue commented on HIVE-8119:
-
[~mohitsabharwal], yes, I'll take a look in
[
https://issues.apache.org/jira/browse/HIVE-8949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298903#comment-14298903
]
Ryan Blue commented on HIVE-8949:
-
[~brocknoland], I lost track of this ticket until
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28933/#review65334
---
Ship it!
Ship It!
- Ryan Blue
On Dec. 15, 2014, 7:40 p.m
onal is always taken.
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
<https://reviews.apache.org/r/28372/#comment107894>
Why not return null here?
- Ryan Blue
On Dec. 12, 2014, 4:03 p.m., Ashish Singh wrote:
>
> -
[
https://issues.apache.org/jira/browse/HIVE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240318#comment-14240318
]
Ryan Blue commented on HIVE-8950:
-
[~singhashish], I think the preferred option is to
nged through
the API. This is why Avro returns an invalid schema that is named something
like "error_error_error_error_error_error_error". Here, maybe we can just
return an empty schema?
- Ryan Blue
On Dec. 7, 2014
ertX method.
Also note that we validate the metadata as we read it from the file, so we
can rely on the fact that if a field is annotated with DECIMAL, its type is
compatible with DECIMAL.
- Ryan Blue
On Dec. 7, 2014, 9:50 p.m., Ashish
[
https://issues.apache.org/jira/browse/HIVE-7787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-7787:
Resolution: Not a Problem
Status: Resolved (was: Patch Available)
I'm closing this per discu
[
https://issues.apache.org/jira/browse/HIVE-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue resolved HIVE-8419.
-
Resolution: Fixed
Assignee: Sergio Peña
I think that this was fixed by Sergio's recent work,
[
https://issues.apache.org/jira/browse/HIVE-7787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236363#comment-14236363
]
Ryan Blue commented on HIVE-7787:
-
[~amalakar], HIVE-8909 recently fixed
[
https://issues.apache.org/jira/browse/HIVE-8949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8949:
Status: Patch Available (was: Open)
Submitting the above patch to get tests to run.
> Test for PARQUET
/reviews.apache.org/r/28372/#comment105678>
This non-functional change is fine with me because it fixes the style in a
method you're editing. But if you add a newline after this conditional, then
you should also add one after line 149.
- Ryan Blue
[
https://issues.apache.org/jira/browse/HIVE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224898#comment-14224898
]
Ryan Blue commented on HIVE-6914:
-
Thanks for fixing the test [~spena]! Looks like i
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223622#comment-14223622
]
Ryan Blue commented on HIVE-6994:
-
[~mickaellcr], has this issue been addressed by
[
https://issues.apache.org/jira/browse/HIVE-8949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8949:
Attachment: StringMapOfOptionalIntArray.parquet
HIVE-8949.1.patch
Adding qtest and source
Ryan Blue created HIVE-8949:
---
Summary: Test for PARQUET-26
Key: HIVE-8949
URL: https://issues.apache.org/jira/browse/HIVE-8949
Project: Hive
Issue Type: Test
Reporter: Ryan Blue
[
https://issues.apache.org/jira/browse/HIVE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-6914:
Attachment: HIVE-6914.3.patch
Updating qtest patch to drop all created tables.
> parquet-hive cannot wr
[
https://issues.apache.org/jira/browse/HIVE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-6914:
Attachment: NestedMap.parquet
HIVE-6914.1.patch
Attaching a qtest and data file that
[
https://issues.apache.org/jira/browse/HIVE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue reassigned HIVE-6914:
---
Assignee: Sergio Peña
> parquet-hive cannot write nested map (map value is
[
https://issues.apache.org/jira/browse/HIVE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221600#comment-14221600
]
Ryan Blue commented on HIVE-6914:
-
HIVE-8909 tests that parquet-hive can now read
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Attachment: HIVE-8909.6.patch
Fixed typo in new map tests (thanks Sergio!)
> Hive doesn't correc
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Attachment: HIVE-8909.5.patch
Fixed projection bug and added tests for map structures.
> Hive does
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Attachment: HIVE-8909.4.patch
Adding patch without the data files.
> Hive doesn't correctly read
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221138#comment-14221138
]
Ryan Blue commented on HIVE-8909:
-
Subversion can handle binaries if they're c
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Attachment: HIVE-8909.3.patch
Updated patch with q tests for all of the cases in TestArrayCompatibility. I
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Attachment: parquet-test-data.tar.gz
I'm attaching a tarball with all of the files from the
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Affects Version/s: 0.13.1
Status: Patch Available (was: Open)
> Hive doesn't correc
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Attachment: HIVE-8909-2.patch
Rebased patch on Sergio's changes. This didn't conflict except for
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218286#comment-14218286
]
Ryan Blue commented on HIVE-8909:
-
Yes. It implements the rules for reading list
[
https://issues.apache.org/jira/browse/HIVE-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14217203#comment-14217203
]
Ryan Blue commented on HIVE-8359:
-
Yes. +1 on this change and we'll clean up the
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14217199#comment-14217199
]
Ryan Blue commented on HIVE-8909:
-
Yes, HIVE-8359 should go in first. This should als
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14217179#comment-14217179
]
Ryan Blue commented on HIVE-8909:
-
[~spena], [~brocknoland], and [~mickaellcr], I t
[
https://issues.apache.org/jira/browse/HIVE-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-8909:
Attachment: HIVE-8909-1.patch
This patch implements the rules from PARQUET-113, which required some
Ryan Blue created HIVE-8909:
---
Summary: Hive doesn't correctly read Parquet nested types
Key: HIVE-8909
URL: https://issues.apache.org/jira/browse/HIVE-8909
Project: Hive
Issue Type
[
https://issues.apache.org/jira/browse/HIVE-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216579#comment-14216579
]
Ryan Blue commented on HIVE-8359:
-
I think with [~mickaellcr]'s addition, this is
should still be in the tests.
- Ryan Blue
On Nov. 14, 2014, 4:06 p.m., Sergio Pena wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.
[
https://issues.apache.org/jira/browse/HIVE-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14212639#comment-14212639
]
Ryan Blue commented on HIVE-8359:
-
[~mickaellcr], it looks HIVE-6994 fixes the read
[
https://issues.apache.org/jira/browse/HIVE-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211431#comment-14211431
]
Ryan Blue commented on HIVE-8359:
-
Thanks, Brock. De-duplicating this and HIVE-6994 i
ect models that will be updated with the
backward-compatibility rules so that it can read parquet-avro and
parquet-thrift structures correctly.
rb
[1]: https://issues.apache.org/jira/browse/PARQUET-113
--
Ryan Blue
Software Engineer
Cloudera, Inc.
tween 1 and 2.
I tried this and the test failed because the null check calls continue,
which skips to the next iteration in writeArray and never calls endGroup().
Adding endGroup() there fixes it; good news!
- Ryan Blue
ing that the key-value type in a
map is named "key_value", that the key is required, etc. It would probably be
worth breaking writeArray and writeMap into separate methods that made
assumptions about the structure of the type, validated those assumptions, and
then wrote out values wi
[
https://issues.apache.org/jira/browse/HIVE-8380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162173#comment-14162173
]
Ryan Blue commented on HIVE-8380:
-
+1 (non-binding)
Looks good to me.
> NanoTim
[
https://issues.apache.org/jira/browse/HIVE-7850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110922#comment-14110922
]
Ryan Blue commented on HIVE-7850:
-
The array fix is something we need to do on the par
[
https://issues.apache.org/jira/browse/HIVE-7850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109347#comment-14109347
]
Ryan Blue commented on HIVE-7850:
-
It looks like {{ArrayWritableGroupConverter}} is
[
https://issues.apache.org/jira/browse/HIVE-7850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109315#comment-14109315
]
Ryan Blue commented on HIVE-7850:
-
Looking at just the changes to the schema conver
[
https://issues.apache.org/jira/browse/HIVE-7459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075497#comment-14075497
]
Ryan Blue commented on HIVE-7459:
-
I don't think there's much chance t
[
https://issues.apache.org/jira/browse/HIVE-7459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated HIVE-7459:
Attachment: 0001-HIVE-7459-Fix-Hive-NPE-when-using-CombineHiveInputFo.patch
Matt Martin's patch, appli
69 matches
Mail list logo
