Tried on some other data sources as well, and it actually works for some
parquet sources. Potentially some specific problems with that first parquet
source that I tried with, and not a Spark 1.4 problem. I'll get back with
more info if I find any new information.
Thanks,
Anders
On Tue, Jun 2,
Just compiled Spark 1.4.0-rc3 for Yarn 2.2 and tried running a job that
worked fine for Spark 1.3. The job starts on the cluster (yarn-cluster
mode), initial stage starts, but the job fails before any task succeeds
with the following error. Any hints?
[ERROR] [06/02/2015 09:05:36.962] [Executor
Does it happen every time you read a parquet source?
On Tue, Jun 2, 2015 at 3:42 AM, Anders Arpteg arp...@spotify.com wrote:
The log is from the log aggregation tool (hortonworks, yarn logs ...),
so both executors and driver. I'll send a private mail to you with the full
logs. Also, tried
How about other jobs? Is it an executor log, or a driver log? Could you
post other logs near this error, please? Thank you.
Best Regards,
Shixiong Zhu
2015-06-02 17:11 GMT+08:00 Anders Arpteg arp...@spotify.com:
Just compiled Spark 1.4.0-rc3 for Yarn 2.2 and tried running a job that
worked
The log is from the log aggregation tool (hortonworks, yarn logs ...), so
both executors and driver. I'll send a private mail to you with the full
logs. Also, tried another job as you suggested, and it actually worked
fine. The first job was reading from a parquet source, and the second from
an
