Re: How to Create one DB connection per executor and close it after the job is done?

VM / Executor. E.g. >> >> object MyDatabseSingleton { >> var dbConn = ??? >> } >> >> On Sat, 28 Jul 2018, 08:34 kant kodali, wrote: >>> >>> Hi All, >>> >>> I understand creating a connection forEachPartition but I am wonderin

Re: How to Create one DB connection per executor and close it after the job is done?

hPartition but I am wondering >> can I create one DB connection per executor and close it after the job is >> done? any sample code would help. you can imagine I am running a simple >> batch processing application. >> >> Thanks! >> >

Re: How to Create one DB connection per executor and close it after the job is done?

ering can > I create one DB connection per executor and close it after the job is done? > any sample code would help. you can imagine I am running a simple batch > processing application. > > Thanks! >

How to Create one DB connection per executor and close it after the job is done?

Hi All, I understand creating a connection forEachPartition but I am wondering can I create one DB connection per executor and close it after the job is done? any sample code would help. you can imagine I am running a simple batch processing application. Thanks!

Re: Create one DB connection per executor

Thanks much Gerard & Manas for your inputs. I'll keep in mind the connection pooling part. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-per-executor-tp26588p26601.html Sent from the Apache Spark User List mailing list arc

Re: Create one DB connection per executor

You are on the right track. The only thing you will have to take care is when two of your partitions try to access the same connection at the same time. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-per-executor-tp26588p26593

Re: Create one DB connection per executor

idate this approach with the spark experts. Does > it have any inherent flaw or is there a better way to create one instance > of > an object per executor? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-pe

Create one DB connection per executor

this approach with the spark experts. Does it have any inherent flaw or is there a better way to create one instance of an object per executor? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-per-executor-tp26588.html Sent from