ach JVM / Executor. E.g.
>>
>> object MyDatabseSingleton {
>> var dbConn = ???
>> }
>>
>> On Sat, 28 Jul 2018, 08:34 kant kodali, wrote:
>>>
>>> Hi All,
>>>
>>> I understand creating a connection forEachPartition but I am won
hPartition but I am wondering
>> can I create one DB connection per executor and close it after the job is
>> done? any sample code would help. you can imagine I am running a simple
>> batch processing application.
>>
>> Thanks!
>>
>
ering can
> I create one DB connection per executor and close it after the job is done?
> any sample code would help. you can imagine I am running a simple batch
> processing application.
>
> Thanks!
>
Hi All,
I understand creating a connection forEachPartition but I am wondering can
I create one DB connection per executor and close it after the job is done?
any sample code would help. you can imagine I am running a simple batch
processing application.
Thanks!
Thanks much Gerard & Manas for your inputs. I'll keep in mind the connection
pooling part.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-per-executor-tp26588p26601.html
Sent from the Apache Spark User List mailing list a
You are on the right track.
The only thing you will have to take care is when two of your partitions try
to access the same connection at the same time.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-per-executor-tp26588p26593
ch with the spark experts. Does
> it have any inherent flaw or is there a better way to create one instance
> of
> an object per executor?
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-per-execut
o validate this approach with the spark experts. Does
it have any inherent flaw or is there a better way to create one instance of
an object per executor?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Create-one-DB-connection-per-executor-tp26588.html
Sen