Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-11-15 Thread Holden Karau
t; Please consider the environment before printing. > > > > > > > > *From: *Li Gao > *Date: *Thursday, November 1, 2018 4:56 > *To: *"Zhang, Yuqi" > *Cc: *Gourav Sengupta , "user@spark.apache.org" > , "Nogami, Masatsugu" > > *S

Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-31 Thread Zhang, Yuqi
ngu...@gmail.com>" mailto:gourav.sengu...@gmail.com>>, "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>>, "Nogami, Masatsugu" Subject: Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how t

Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-31 Thread Li Gao
ts. Instead, please > notify the sender and delete the e-mail and any attachments. Thank you. > > Please consider the environment before printing. > > > > > > > > *From: *Li Gao > *Date: *Thursday, November 1, 2018 0:07 > *To: *"Zhang, Yuqi" > *

Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-31 Thread Zhang, Yuqi
Please consider the environment before printing. From: Gourav Sengupta mailto:gourav.sengu...@gmail.com>> Date: Wednesday, October 31, 2018 18:34 To: "Zhang, Yuqi" Cc: user mailto:user@spark.apache.org>>, "Nogami, Masatsugu" Subject: Re: [Spark Shell on AWS K8

Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-31 Thread Li Gao
ts. Instead, please > notify the sender and delete the e-mail and any attachments. Thank you. > > Please consider the environment before printing. > > > > > > > > *From: *Gourav Sengupta > *Date: *Wednesday, October 31, 2018 18:34 > *To: *"Zhang, Yuqi"

Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-31 Thread Zhang, Yuqi
Instead, please notify the sender and delete the e-mail and any attachments. Thank you. Please consider the environment before printing. From: Gourav Sengupta Date: Wednesday, October 31, 2018 18:34 To: "Zhang, Yuqi" Cc: user , "Nogami, Masatsugu" Subject: Re: [Spark Shel

Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-31 Thread Biplob Biswas
Hi Yuqi, Just curious can you share the spark-submit script and what are you passing as --master argument? Thanks & Regards Biplob Biswas On Wed, Oct 31, 2018 at 10:34 AM Gourav Sengupta wrote: > Just out of curiosity why would you not use Glue (which is Spark on > kubernetes) or EMR? > >

Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-31 Thread Gourav Sengupta
Just out of curiosity why would you not use Glue (which is Spark on kubernetes) or EMR? Regards, Gourav Sengupta On Mon, Oct 29, 2018 at 1:29 AM Zhang, Yuqi wrote: > Hello guys, > > > > I am Yuqi from Teradata Tokyo. Sorry to disturb but I have some problem > regarding using spark 2.4 client

[Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster?

2018-10-28 Thread Zhang, Yuqi
Hello guys, I am Yuqi from Teradata Tokyo. Sorry to disturb but I have some problem regarding using spark 2.4 client mode function on kubernetes cluster, so I would like to ask if there is some solution to my problem. The problem is when I am trying to run spark-shell on kubernetes v1.11.3