big data products

Hello Currently my company uses tech stacks for big data, besides hadoop, it includes Hive for common SQL jobs, Spark for batching jobs, and Flink for streaming jobs. Besides that, what are other useful products for big-data jobs? Thanks for your sharing. regards.

Re: Spark with GPU

As far as I understand you will need a GPU for each worker node or you will need to partition the GPU processing somehow to each node which I think would defeat the purpose. In Databricks for example when you select GPU workers there is a GPU allocated to each worker. I assume this is the “correct”

Re: Spark with GPU

if you have several nodes with only one node having GPUs, you still have to wait for the result set to complete. In other words it will be as fast as the lowest denominator .. my postulation HTH view my Linkedin profile https://e

Spark with GPU

Hello, has anyone used spark with GPUs? I wonder if every worker node in a cluster needs one GPU or if you can have several worker nodes of which only one has a GPU. Thank you! - To unsubscribe e-mail: user-unsubscr...@spa