I tried
pig --param "input=s3n://bucket_path/*/" test.pig
It worked for me. I am on EMR Pig 0.9.1.
Thanks,
Aniket
On Tue, Dec 27, 2011 at 3:35 PM, Corbin Hoenes wrote:
> I am not sure Ayon doesn't have something here. I am seeing a similar
> problem with the 0.9.1 build of pig. But when I ru
I am not sure Ayon doesn't have something here. I am seeing a similar problem
with the 0.9.1 build of pig. But when I run with 0.9.0 it doesn't have that
problem.
Did something with pattern substitution change from 0.9.0 --> 0.9.1? Haven't
run it through a debugger yet but that is the next
Thanks Daniel, I will create a jira and submit a patch soon.
I was wondering if there is a way to "by default" register(define) a set of
jars and macros everytime pig command is run. Please let me know.
AFAIK, for jars, I can put them into pig.additional.jars and for params
(defines) I can do --p
Actually I fixed it. I had to use an additional grouping to make it really
Algebraic. But now I see OutOfMemory during Map merge:
[2011-12-27 08:44:07] FATAL (Child.java:318) - Error running child :
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.IFile$Reader.read
