unsubscribe

[Daniel Tavares de Santana]

unsubscribe

Re: [EXTERNAL] Use of ML in certain aspects of Spark to improve the performance

[Daniel Tavares de Santana]

unsubscribe

From: Mich Talebzadeh 
Sent: Tuesday, August 8, 2023 4:43 PM
To: user @spark 
Subject: [EXTERNAL] Use of ML in certain aspects of Spark to improve the 
performance

I am currently pondering and sharing my thoughts openly. Given our reliance on 
gathered statistics, it prompts the question of whether we could integrate 
specific machine learning components into Spark Structured Streaming. Consider 
a scenario

I am currently pondering and sharing my thoughts openly. Given our reliance on 
gathered statistics, it prompts the question of whether we could integrate 
specific machine learning components into Spark Structured Streaming. Consider 
a scenario where we aim to adjust configuration values on the fly based on 
collected statistics. For instance, Spark GUI could gather some statistics via 
Spark listeners. Is this a viable approach? This aligns with use cases 
involving Spark on Kubernetes autopilot and Spark Structured Streaming. While 
this might slightly veer from our usual discussions, I admit my lack of 
expertise in machine learning. However, there are experts among us who could 
lend their knowledge to explore this potential avenue.

Mich Talebzadeh,
Solutions Architect/Engineering Lead
London
United Kingdom


 
[https://ci3.googleusercontent.com/mail-sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE]
   view my Linkedin 
profile


 
https://en.everybodywiki.com/Mich_Talebzadeh



Disclaimer: Use it at your own risk. Any and all responsibility for any loss, 
damage or destruction of data or any other property which may arise from 
relying on this email's technical content is explicitly disclaimed. The author 
will in no case be liable for any monetary damages arising from such loss, 
damage or destruction.

Re: [EXTERNAL] Spark Not Connecting

[Daniel Tavares de Santana]

unsubscribe

From: timi ayoade 
Sent: Wednesday, July 12, 2023 6:11 AM
To: user@spark.apache.org 
Subject: [EXTERNAL] Spark Not Connecting

Hi Apache spark community, I am a Data EngineerI have been using Apache spark 
for some time now. I recently tried to use it but I have been getting some 
errors. I have tried debugging the error but to no avail. the screenshot is 
attached below. 

Hi Apache spark community, I am a Data EngineerI have been using Apache spark 
for some time now. I recently tried to use it but I have been getting some 
errors. I have tried debugging the error but to no avail. the screenshot is 
attached below. I will be glad if responded to. thanks

unsubscribe

[Daniel Tavares de Santana]

unsubscribe