The only way I can think of is through some kind of wrapper. For java/scala,
use JNI. For Python, use extensions. There should not be a lot of work if you
know these tools.
From: Robin East
To: Annabel Melongo
Cc: Jia
Thanks Josh and Marcelo! It now works!
BTW, just wondering, is there any perf difference between running spark in
standalone mode and under yarn? The only goal that I created this cluster is to
run spark jobs. So I can set up spark in standalone mode if it runs slow in
yarn.
best.
From: