Interesting.
Spark supports the following cluster managers
- Standalone: A cluster-manager, limited in features, shipped with Spark.
- Apache Hadoop YARN is the most widely used resource manager not just
for Spark but for other artefacts as well. On-premise YARN is used
extensively.
Good afternoon.
Perhaps you will be discouraged by what I will write below, but nevertheless, I
ask for help in solving my problem. Perhaps the architecture of our solution
will not seem correct to you.
There are backend services that communicate with a service that implements
spark-driver.
