[jira] [Created] (PIG-2837) AvroStorage throws StackOverFlowError

2012-07-25 Thread Mubarak Seyed (JIRA)
Mubarak Seyed created PIG-2837: -- Summary: AvroStorage throws StackOverFlowError Key: PIG-2837 URL: https://issues.apache.org/jira/browse/PIG-2837 Project: Pig Issue Type: Bug Component

[jira] [Commented] (PIG-2779) Refactoring the code for setting number of reducers

2012-07-25 Thread Jie Li (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422762#comment-13422762 ] Jie Li commented on PIG-2779: - estimatedParallelism is a good one to add and these names look go

writing info to configs

2012-07-25 Thread Bill Graham
Hi, Pig internally generates a lot of runtime info that would be useful to have output in the job config for later debugging analysis. Should we develop a standard for params are written to the job conf file, but not accepted as job input? For example, these params are accepted as input: pig.ex

Re: Number of mappers in MRCompiler

2012-07-25 Thread Prasanth J
Thanks Alan. The requirement for me is that I want to load N number of samples based on the input file size and perform naive cube computation to determine the large groups that will not fit in reducer's memory. I need to know the exact number of samples for calculating the partition factor for

Re: Number of mappers in MRCompiler

2012-07-25 Thread Alan Gates
No. The number of mappers is determined by the InputFormat used by your load function (TextInputFormat if you're using the default PigStorage loader) when the Hadoop job is submitted. Pig doesn't have access to that info until it's handed the jobs off to MapReduce. Alan. On Jul 25, 2012, at

Number of mappers in MRCompiler

2012-07-25 Thread Prasanth J
Hello everyone I would like know if there is a way to know the number of mappers while compiling physical plan to MR-plan. Thanks -- Prasanth

[jira] [Commented] (PIG-2779) Refactoring the code for setting number of reducers

2012-07-25 Thread Bill Graham (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422680#comment-13422680 ] Bill Graham commented on PIG-2779: -- Great! We should probably set {{estimatedParallelism}}

[jira] [Assigned] (PIG-2579) Support for multiple input schemas in AvroStorage

2012-07-25 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park reassigned PIG-2579: -- Assignee: Cheolsoo Park > Support for multiple input schemas in AvroStorage > ---

[jira] [Commented] (PIG-2729) Macro expansion does not use pig.import.search.path - UnitTest borked

2012-07-25 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422496#comment-13422496 ] Rohini Palaniswamy commented on PIG-2729: - Thanks Johannes. My +1. I will ask Dani

Re: Reviewboard gives Error 500

2012-07-25 Thread Prasanth J
Is your patch generated using git? If so try using "pig-git" trunk instead of pig trunk. Thanks -- Prasanth On Jul 25, 2012, at 4:53 AM, Johannes Schwenk wrote: > The review boards seems to have an issue with my requests. I always get > > -- > Something broke! (Error 500) > > It appears somet

[jira] [Commented] (PIG-2779) Refactoring the code for setting number of reducers

2012-07-25 Thread Jie Li (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422459#comment-13422459 ] Jie Li commented on PIG-2779: - Great point Bill! Yeah I'll set defaultParallel and requestedPara

Reviewboard gives Error 500

2012-07-25 Thread Johannes Schwenk
The review boards seems to have an issue with my requests. I always get -- Something broke! (Error 500) It appears something broke when you tried to go to here. This is either a bug in Review Board or a server configuration error. Please report this to your administrator. -- I just wanted to pos

[jira] [Updated] (PIG-2729) Macro expansion does not use pig.import.search.path - UnitTest borked

2012-07-25 Thread Johannes Schwenk (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johannes Schwenk updated PIG-2729: -- Attachment: PIG-2729.patch I included your corrections and good advice. I think the patch has evo