[jira] [Commented] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()
[
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473535#comment-13473535
]
Thejas M Nair commented on PIG-2910:
Yes, The changes in 2910-3 patch look good. Can you please add a test case, and
also add a comment that this schema string has "{}" around it, and that is why
the substring is being done ?
> Make toString() methods on Schema and FieldSchema be readable by
> Utils.getSchemaFromString()
>
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
> Issue Type: Bug
> Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Thejas M Nair
> Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch, PIG-2910-3.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At
> the moment this requires writing your own toString() method that
> Utils.getSchemaFromString() can read. Making a readable schema for the
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1,
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()
[
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472941#comment-13472941
]
Thejas M Nair commented on PIG-2910:
The extra parenthesis added by Schema.toString() are curly braces. I believe
this is because the schema when thought of as a schema of a relation, is a
schema of a bag. I don't think the PIG-2910-2.patch will fix the issue.
But changing the behavior of either Schema.toString() or
Utils.getSchemaFromString() will break backward compatibility. I think we
should just add a new function Utils.getSchemaFromBagSchemaString() and comment
that people should use this one to get schema back from output of
Schema.toString().
The use of input schema of udf, during udf execution is not very common. I
don't think we should serialize it for all udfs.
> Make toString() methods on Schema and FieldSchema be readable by
> Utils.getSchemaFromString()
>
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
> Issue Type: Bug
> Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Thejas M Nair
> Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At
> the moment this requires writing your own toString() method that
> Utils.getSchemaFromString() can read. Making a readable schema for the
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1,
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()
[
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471904#comment-13471904
]
Eli Reisman commented on PIG-2910:
--
I can fix it in the toString() instead. Also, if its OK to publish use of
UDFContext into the Schema namespace, I can put the hardcoded properties name
String and the context itself inside the method call to keep client code from
knowing how this transfer of schema to the Properties is implemented? Sound
good?
I have not sat down and poured over the Pig code yet the way I need to so I can
recognize the optimal solution for some of these JIRA's yet off the top of my
head. I was not aware whether removing the outer indication of the TUPLE or BAG
container in the toString() would break other code, or just plain confuse
people who use toString() on Schemas for their own reasons. Anyway, a serious
"Pig study weekend" needs to be next on my list before I get to put up too many
more patches!
> Make toString() methods on Schema and FieldSchema be readable by
> Utils.getSchemaFromString()
>
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
> Issue Type: Bug
> Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Thejas M Nair
> Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At
> the moment this requires writing your own toString() method that
> Utils.getSchemaFromString() can read. Making a readable schema for the
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1,
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()
[
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470633#comment-13470633
]
Russell Jurney commented on PIG-2910:
-
In looking at the toString code, it looked like a () was added around both bags
and tuples - without checking the type. Thejas was able to reproduce and
address the problem. It might be better to fix the code at the source of the
issue - the toString method but... this works for me. :)
> Make toString() methods on Schema and FieldSchema be readable by
> Utils.getSchemaFromString()
>
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
> Issue Type: Bug
> Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Thejas M Nair
> Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At
> the moment this requires writing your own toString() method that
> Utils.getSchemaFromString() can read. Making a readable schema for the
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1,
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
