[
https://issues.apache.org/jira/browse/PIG-505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642326#action_12642326
]
David Ciemiewicz commented on PIG-505:
--------------------------------------
Just tried naively changing Map<Object,Object> to Map<String,String> for
string.URLPARSE and realized that this won't work.
The thing is that some of the mapped values are strings and some are maps such
as paramsFields and queryFields.
When you look at the code, it will make sense.
So the type must be "unknown" and not "bytearray" because it could be String or
Map.
So, it would seem that there is need for "unknown" but, in this case, the cast
operation is not a bytearray to whatever conversion but is rather Object
(unknown) to Type (Map) with an opaque, no conversion cast of the data.
urlparse
#'scheme' String
#'authority' String
#'userinfo' String
#'host' String
#'port' String
#'path' String
#'params' String
#'paramsFields' Map
#'query' String
#'queryFields' Map
#'fragment' String
> Lineage for UDFs that do not return bytearray
> ---------------------------------------------
>
> Key: PIG-505
> URL: https://issues.apache.org/jira/browse/PIG-505
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Santhosh Srinivasan
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
>
> In Pig-335, the lineage design states that UDFs that return bytearrays could
> cause problems in tracing the lineage. For UDFs that do not return bytearray,
> the lineage design should pickup the right load function to use as long as
> there is no ambiguity. In the current implementation, we could have issues
> with scripts like:
> {code}
> a = load 'input' as (field1);
> b = foreach a generate myudf_to_double(field1);
> c = foreach b generate $0 + 2.0;
> {code}
> When $0 has to be cast to a double, the lineage code will complain that it
> hit a UDF and hence cannot determine the right load function to use.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
