Re: Improving macros

2013-06-20 Thread Jonathan Packer
Submitted a patch for this: https://issues.apache.org/jira/browse/PIG-3359 On Wed, Jun 19, 2013 at 5:36 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Jon is right. I am trying to ensure that each line is mostly parsed only once in https://issues.apache.org/jira/browse/PIG-3204. Have

Improving macros

2013-06-19 Thread Jonathan Packer
Hi, I'm an engineer at Mortar Data. I was working on some features to improve macros that I'd like to contribute (we're hoping to build a library of reusable pig macros implementing common algorithms), but I wanted to check-in here first to see if anyone has concerns about the changes I'd be

Re: Improving macros

2013-06-19 Thread Jonathan Coveney
This sounds excellent! Would love to see this in trunk. As far as #3, this is probably because pig does essentially reparse everything with each new line. I know there is a ticket where Rohini dealt with this in some cases where HDFS was being hit multiple times because of load statements getting

Re: Improving macros

2013-06-19 Thread Rohini Palaniswamy
Jon is right. I am trying to ensure that each line is mostly parsed only once in https://issues.apache.org/jira/browse/PIG-3204. Have few issues with other commands in pig script like fs, shell, cd, illustrate, error messages not showing line numbers properly, etc which I have not got to solving