sql("SELECT * FROM ").foreach(println)
can be executed successfully. So the problem may still be in UDF code. How
can i print the the line with ArrayIndexOutOfBoundsException in catalyst?
2015-03-23 17:04 GMT+08:00 lonely Feb :
> ok i'll try asap
>
> 2015-03-23 17
see similar exception.
> And you can catch it with the help of the output. Notice that the messages
> are printed to stdout on executor side.
>
> On 3/23/15 4:36 PM, lonely Feb wrote:
>
> I caught exceptions in the python UDF code, flush exceptions into a
> single file, and
java.lang.ArrayIndexOutOfBoundsException: 9, is
there any inspirations?
2015-03-23 16:24 GMT+08:00 Cheng Lian :
> Could you elaborate on the UDF code?
>
>
> On 3/23/15 3:43 PM, lonely Feb wrote:
>
>> Hi all, I tried to transfer some hive jobs into spark-sql. When i ran a
>> sql job with p
Hi all, I tried to transfer some hive jobs into spark-sql. When i ran a sql
job with python udf i got a exception:
java.lang.ArrayIndexOutOfBoundsException: 9
at
org.apache.spark.sql.catalyst.expressions.GenericRow.apply(Row.scala:142)
at
org.apache.spark.sql.catalyst.expressions.B
+spark-user
-- Forwarded message --
From: lonely Feb
Date: 2015-01-16 19:09 GMT+08:00
Subject: Re: Problems with TeraValidate
To: Ewan Higgs
thx a lot.
btw, here is my output:
1. when dataset is 1000g:
num records: 100
checksum: 12aa5028310ea763e
part 0
Hi all , i tried to run a terasort benchmark on my spark cluster, but i
found it is hard to find a standard spark terasort program except a PR from
rxin and ewan higgs:
https://github.com/apache/spark/pull/1242
https://github.com/ehiggs/spark/tree/terasort
The example which rxin provided without
Hi all, i sadly found on YARN mode i cannot view executor logs on YARN web
UI nor on SPARK history web UI. On YARN web UI i can only view AppMaster
logs and on SPARK history web UI i can only view Application metric
information. If i want to see whether a executor is being full GC i can
only use "y
