r serverless is 60 seconds compared
>> to dataproc on Compute engine (the one you setup your own spark cluster on
>> dataproc tin boxes) of 90 seconds
>>
>> Dataproc serverless for Spark autoscaling
>> <https://cloud.google.com/dataproc-serverless/docs/c
k application itself.
> Reading the doc it says startup time for serverless is 60 seconds compared
> to dataproc on Compute engine (the one you setup your own spark cluster on
> dataproc tin boxes) of 90 seconds
>
> Dataproc serverless for Spark autoscaling
> <https://cloud.google.com
u setup your own spark cluster on
dataproc tin boxes) of 90 seconds
Dataproc serverless for Spark autoscaling
<https://cloud.google.com/dataproc-serverless/docs/concepts/autoscaling> makes
a reference to "Dataproc Serverless autoscaling is the default behavior,
and uses Spark dynamic res
Out of curiosity : are there functional limitations in Spark Standalone
that are of concern? Yarn is more configurable for running non-spark
workloads and how to run multiple spark jobs in parallel. But for a single
spark job it seems standalone launches more quickly and does not miss any
Hi,
I have not tested this myself but Google have brought up *Dataproc Serverless
for Spar*k. in a nutshell Dataproc Serverless lets you run Spark batch
workloads without requiring you to provision and manage your own cluster.
Specify workload parameters, and then submit the workload to the
