[jira] [Commented] (HIVE-3323) Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440666#comment-13440666
]
Feng Peng commented on HIVE-3323:
-
Bump.
Is it possible some committer to take a look at this patch? A few of our later
patches depend on it and it'd be great if we can get some feedback. Thanks!
Enable enum to string conversions
-
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13428315#comment-13428315
]
Feng Peng commented on HIVE-3323:
-
The previous patch didn't cover nested structures in map values. Posted a new
patch using a static property in ObjectInspectorFactory instead of passing the
property around. A better solution would be making the ObjectInspectorFactory a
singleton instead of the current stateless static class.
Enable enum to string conversions
-
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13428474#comment-13428474
]
Travis Crawford commented on HIVE-3323:
---
Hey all -
Feng and I looked at this today and decided we missed the nested map value case
because we didn't have a good way to test. We have an update (running CI now,
will post patch after it passes) that adds a new test struct that contains all
thrift data types, as well as some gnarly cases we found in real data (enum map
keys, enum map values, etc).
https://github.com/traviscrawford/hive/blob/HIVE-3323_enum_to_string/serde/if/test/megastruct.thrift
This new struct will allow us to test this and other thrift cases, such as how
binary data is not handled correctly (its interpreted as a ByteBuffer struct
instead of byte[]).
Will post what we believe is the last version of this patch after tests finish.
Enable enum to string conversions
-
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426779#comment-13426779
]
Travis Crawford commented on HIVE-3323:
---
I'm not able to post the review with arc, getting the following error:
{code}
TW-MBP13-TCrawford:hive travis$ arc diff --jira HIVE-3323
Exception:
Parse Exception: Expected a hunk header, like 'Index: /path/to/file.ext' (svn),
'Property changes on: /path/to/file.ext' (svn properties), 'commit
59bcc3ad6775562f845953cf01624225' (git show), 'diff --git' (git diff), or '---
filename' (unified diff).
1
(Run with --trace for a full exception trace.)
TW-MBP13-TCrawford:hive travis$
{code}
Enable enum to string conversions
-
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426868#comment-13426868
]
Jakob Homan commented on HIVE-3323:
---
Just as another data point, the AvroSerde already converts Avro enums to
strings, so this will make the behavior more consistent.
Enable enum to string conversions
-
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426259#comment-13426259
]
Travis Crawford commented on HIVE-3323:
---
Summary of what this feature enables:
{code}
hive create table convert_enum_to_string
partitioned by (b string)
row format serde
org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer
with serdeproperties (
serialization.class=org.apache.hadoop.hive.serde2.thrift.test.Complex,
serialization.format=org.apache.thrift.protocol.TBinaryProtocol);
OK
Time taken: 0.067 seconds
hive set hive.data.convert.enum.to.string=true;
hive describe convert_enum_to_string;
OK
aintint from deserializer
astring string from deserializer
lintarrayint from deserializer
lstring arraystring from deserializer
lintstring arraystructmyint:int,mystring:string,underscore_int:int
from deserializer
mstringstring mapstring,string from deserializer
myenum string from deserializer
b string
Time taken: 0.349 seconds
hive set hive.data.convert.enum.to.string=false;
hive describe convert_enum_to_string;
OK
aintint from deserializer
astring string from deserializer
lintarrayint from deserializer
lstring arraystring from deserializer
lintstring arraystructmyint:int,mystring:string,underscore_int:int
from deserializer
mstringstring mapstring,string from deserializer
myenum structvalue:int from deserializer
b string
Time taken: 0.144 seconds
{code}
Enable enum to string conversions
-
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
