[
https://issues.apache.org/jira/browse/PIG-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837574#action_12837574
]
Daniel Dai commented on PIG-1016:
---------------------------------
Hi, busy,
I checked your code, seems your patch assume PIG-1016.patch checked in. If I
understand correctly, there are inconsistency in this approach. In your code,
you allow map value to be any type. However, internally Pig always assume map
value to be bytearray. So Pig will choose to use PigBytesRawComparator. And you
further modify PigBytesRawComparator to handle all data type. This logic is
very confusing. Further, TextDataParser itself if bogus since it will guess the
data type based on the content.
In PIG-613, we reiterate that map value is bytearray. However, we fixed the
code which can cast bytearray to map/tuple/bag correctly. I verified the test
case you gave, and it works.
{code}
A= load '9.txt' as (data:map[]);
B= foreach A generate (int)(data#'a'),
(chararray)(data#'b'),(tuple(map[]))(data#'c');
C= order B by $0;
dump C;
{code}
Result:
(1,'a',(1,2,3))
(2,'d',(1,2,3))
(3,'c',(1,2,3))
{code}
D= order B by $1;
dump D;
{code}
Result:
(1,'a',(1,2,3))
(3,'c',(1,2,3))
(2,'d',(1,2,3))
{code}
describe B;
{code}
Result:
B: {int,chararray,(map[ ])}
Do you have other use cases which PIG-613 cannot address?
> Reading in map data seems broken
> --------------------------------
>
> Key: PIG-1016
> URL: https://issues.apache.org/jira/browse/PIG-1016
> Project: Pig
> Issue Type: Improvement
> Components: data
> Affects Versions: 0.4.0
> Reporter: hc busy
> Attachments: PIG-1016.patch
>
>
> Hi, I'm trying to load a map that has a tuple for value. The read fails in
> 0.4.0 because of a misconfiguration in the parser. Where as in almost all
> documentation it is stated that value of the map can be any time.
> I've attached a patch that allows us to read in complex objects as value as
> documented. I've done simple verification of loading in maps with tuple/map
> values and writing them back out using LOAD and STORE. All seems to work fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
