Re: [Spark Core] [Advanced] [How-to] How to map any external field to job ids spawned by Spark.

Hi Khalid, just out of curiosity, does the API help us in setting JOB ID's or just job Descriptions? Regards, Gourav Sengupta On Wed, Dec 28, 2022 at 10:58 AM Khalid Mammadov wrote: > There is a feature in SparkContext to set localProperties > (setLocalProperty) where you can set your Request

Re: [Spark Core] [Advanced] [How-to] How to map any external field to job ids spawned by Spark.

There is a feature in SparkContext to set localProperties (setLocalProperty) where you can set your Request ID and then using SparkListener instance read that ID with Job ID using onJobStart event. Hope this helps. On Tue, 27 Dec 2022, 13:04 Dhruv Toshniwal, wrote: > TL;Dr - >

[Spark Core] [Advanced] [How-to] How to map any external field to job ids spawned by Spark.

TL;Dr - how-to-map-external-request-ids-to-spark-job-ids-for-spark-instrumentation Hi team, We are the engineering team of Mindtickle Inc. and we have a use-case where we