[jira] [Commented] (PIG-1890) Fix piggybank unit test TestAvroStorage
[
https://issues.apache.org/jira/browse/PIG-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13060767#comment-13060767
]
Mads Moeller commented on PIG-1890:
---
Hi Ken,
With the latest patch the UNION behaves as expected for me.
Thanks,
Mads
Fix piggybank unit test TestAvroStorage
---
Key: PIG-1890
URL: https://issues.apache.org/jira/browse/PIG-1890
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Daniel Dai
Assignee: Jakob Homan
Attachments: PIG-1890-1.patch, PIG-1890-2.patch, PIG-1890-3.patch,
pig_setloc_avro.txt
TestAvroStorage fail on trunk. There are two reasons:
1. After PIG-1680, we call LoadFunc.setLocation one more time.
2. The schema for AvroStorage seems to be wrong. For example, in first test
case testArrayDefault, the schema for in is set to PIG_WRAPPER: (FIELD:
{PIG_WRAPPER: (ARRAY_ELEM: float)}). It seems PIG_WRAPPER is redundant. This
issue is hidden until PIG-1188 checked in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1890) Fix piggybank unit test TestAvroStorage
[
https://issues.apache.org/jira/browse/PIG-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13060190#comment-13060190
]
Mads Moeller commented on PIG-1890:
---
Re-pasting addInputPaths.
{code}
/**
* get input paths to job config
*/
public static boolean addInputPaths(String pathString, Job job)
throws IOException {
SetPath pathSet = new HashSetPath();
if (addAllSubDirs(new Path(pathString), job, pathSet)) {
Path[] paths = pathSet.toArray(new Path[pathSet.size()]);
FileInputFormat.setInputPaths(job, paths);
return true;
}
return false;
}
{code}
Fix piggybank unit test TestAvroStorage
---
Key: PIG-1890
URL: https://issues.apache.org/jira/browse/PIG-1890
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Daniel Dai
Assignee: Jakob Homan
Attachments: PIG-1890-1.patch, PIG-1890-2.patch
TestAvroStorage fail on trunk. There are two reasons:
1. After PIG-1680, we call LoadFunc.setLocation one more time.
2. The schema for AvroStorage seems to be wrong. For example, in first test
case testArrayDefault, the schema for in is set to PIG_WRAPPER: (FIELD:
{PIG_WRAPPER: (ARRAY_ELEM: float)}). It seems PIG_WRAPPER is redundant. This
issue is hidden until PIG-1188 checked in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2036) Set header delimiter in PigStorageSchema
[ https://issues.apache.org/jira/browse/PIG-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13029713#comment-13029713 ] Mads Moeller commented on PIG-2036: --- Dmitriy, Thanks for going over the patch. Your changes makes it much cleaner. Set header delimiter in PigStorageSchema Key: PIG-2036 URL: https://issues.apache.org/jira/browse/PIG-2036 Project: Pig Issue Type: Improvement Components: impl Affects Versions: 0.7.0, 0.8.0 Reporter: Mads Moeller Assignee: Mads Moeller Priority: Minor Attachments: PIG-2036.1.patch, PIG-2036.patch Piggybanks' PigStorageSchema currently defaults the delimiter to a tab in the generated header file (.pig_header). The attached patch set the header delimiter to what is passed in via the constructor. Otherwise it'll default to tab '\t'. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2036) Set header delimiter in PigStorageSchema
[ https://issues.apache.org/jira/browse/PIG-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mads Moeller updated PIG-2036: -- Attachment: PIG-2036.patch Patch Set header delimiter in PigStorageSchema Key: PIG-2036 URL: https://issues.apache.org/jira/browse/PIG-2036 Project: Pig Issue Type: Improvement Components: impl Affects Versions: 0.7.0, 0.8.0 Reporter: Mads Moeller Priority: Minor Attachments: PIG-2036.patch Piggybanks' PigStorageSchema currently defaults the delimiter to a tab in the generated header file (.pig_header). The attached patch set the header delimiter to what is passed in via the constructor. Otherwise it'll default to tab '\t'. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
