I am using 0.7.0...perhaps when I can get 0.8.0 it will fix the issue. The
script is not the script I run -- I just realiased the actual script, so the
issues you raises, while valid, come from that and are not an issue in the
actual script.
Logic wise, however, should it work?
2010/12/15 Daniel
Yes, you can use EvalFunc.warn(Object o, String msg, Enum warningEnum).
Daniel
Dexin Wang wrote:
Is it possible to increment a counter in Pig UDF (in either Load/Eval/Store
Func).
Since we have access to counters using the
org.apache.hadoop.mapred.Reporter:
http://hadoop.apache.org/common/doc
Is it possible to increment a counter in Pig UDF (in either Load/Eval/Store
Func).
Since we have access to counters using the
org.apache.hadoop.mapred.Reporter:
http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Counters
the other way to ask this question is how do we get an insta
Which version of Pig are you using? I find some syntax error in your
script. Is this the script you actually run?
Here is the syntax error I find:
1. What is ahh, ooh?
2. Alias cannot be "group", it is a keyword
3. "sort = ORDER counts BY cnt DESC; ". Do you mean "sort = ORDER count
BY cnt DESC
The outputSchema is set to Long
90 @Override
91 public Schema outputSchema(Schema input) {
92 return new Schema(new
Schema.FieldSchema(getSchemaName(this.getClass
().getName().toLowerCase(), input), DataType.CHARARRAY));
93 }
The describe output is
eventData: {event: cha
Yes it's tough, and no it's not common :)
Scale brings limitations...
On Tue, Dec 14, 2010 at 4:05 AM, Rajesh Balamohan
wrote:
> Thanks for the quick reply Dmitriy. Does it mean that its tough to have
> non-equi join type of joins between 2 datasets in PIG? Isn't it a common
> scenario in product
What is the describe output?
Are you setting the outputSchema appropriately in ISOToUnixInSecond ?
(btw.. you probably want to call that InSeconds, plural)
D
On Wed, Dec 15, 2010 at 2:00 PM, John Hui wrote:
> To give more context, the ISOToUnixInSecond return UnixTime in second. The
> return
To give more context, the ISOToUnixInSecond return UnixTime in second. The
return value of this function is Long
75 @Override
76 public Long exec(Tuple input) throws IOException
77 {
78 if (input == null || input.size() < 1) {
79 return null;
80 }
81
I am getting an error I have not seen before and would love some help. I did
a DESCRIBE and it parses fine, but when you actually try and execute, that
is when it blows up.
Here is the error:
2010-12-15 16:25:33,084 [main] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapRedu
This is actually, please ignore the code section below, Thanks!
7 eventData = FOREACH flattenData GENERATE B1::event AS event,
B1::publication AS publication, B1::deviceType AS deviceType, B1::adID AS
adID, B1::mcc AS mcc, B1::event_timestamp AS eventTimestamp:long,
ISOToUnixInSecond('$STARTDATETI
I am having a hard time getting comparison to work. I am comparing from two
long values but I keep on getting a cast long to String error
Backend error message
-
java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.String
at java.lang.String.compareT
Excellent, that did the trick.
For reference, I did:
export PIG_OPTS="$PIG_OPTS -Dmapred.max.split.size=100"
Thanks for your help.
- Charles
On Tue, Dec 14, 2010 at 11:59 PM, Dmitriy Ryaboy wrote:
> Try
>
> set mapred.max.split.size $desired_split_size
>
> -D
>
> On Tue, Dec 14, 2010 at
PIG-1769 has been created sorry I lost track of this :(
On Nov 22, 2010, at 2:30 PM, Dmitriy Ryaboy wrote:
> Hm, good point. Can you create a JIRA for this?
>
> On Mon, Nov 22, 2010 at 1:16 PM, Corbin Hoenes wrote:
>
>> One comment on the HBaseStorage store func. In our load statement we
>> a
13 matches
Mail list logo
