Re: Re: spark+kafka+dynamic resource allocation

2023-01-28 Thread Lingzhe Sun
Thank you for the response. But the reference does not seem to be answering any of those questions. BS Lingzhe Sun From: ashok34...@yahoo.com Date: 2023-01-29 04:01 To: User; Lingzhe Sun Subject: Re: spark+kafka+dynamic resource allocation Hi, Worth checking this link

Fwd: Spark-submit doesn't load all app classes in the classpath

2023-01-28 Thread Soheil Pourbafrani
Hello all, I'm using Oozie to manage a Spark application on YARN cluster, in yarn-cluster mode. Recently I made some changes to the application in which the Hikari lib was involved. Surprisingly when I started the job, I got ClassNotFound exception for the Hikari classes. I'm passing a shade jar

Re: spark+kafka+dynamic resource allocation

2023-01-28 Thread ashok34...@yahoo.com.INVALID
Hi, Worth checking this link https://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation On Saturday, 28 January 2023 at 06:18:28 GMT, Lingzhe Sun wrote: #yiv9684413148 body {line-height:1.5;}#yiv9684413148 ol, #yiv9684413148 ul

Re: Spark SQL question

2023-01-28 Thread Bjørn Jørgensen
Hi Mich. This is a Spark user group mailing list where people can ask *any* questions about spark. You know SQL and streaming, but I don't think it's necessary to start a replay with "*LOL*" to the question that's being asked. No questions are to stupid to be asked. lør. 28. jan. 2023 kl. 09:22

Re: Spark SQL question

2023-01-28 Thread Mich Talebzadeh
LOL First one spark-sql> select 1 as `data.group` from abc group by data.group; 1 Time taken: 0.198 seconds, Fetched 1 row(s) means that are assigning alias data.group to select and you are using that alias -> data.group in your group by statement This is equivalent to spark-sql> select 1