[mailto:so...@cloudera.com]
Sent: Friday, February 13, 2015 4:46 PM
To: Igor Petrov
Cc: user@spark.apache.org
Subject: Re: Tuning number of partitions per CPU
18 cores or 36? doesn't probably matter.
For this case where you have some overhead per partition of setting up the
DB connection, it may
18 cores or 36? doesn't probably matter.
For this case where you have some overhead per partition of setting up
the DB connection, it may indeed not help to chop up the data more
finely than your total parallelism. Although that would imply quite an
overhead. Are you doing any other expensive
, February 13, 2015 4:46 PM
To: Igor Petrov
Cc: user@spark.apache.org
Subject: Re: Tuning number of partitions per CPU
18 cores or 36? doesn't probably matter.
For this case where you have some overhead per partition of setting up the DB
connection, it may indeed not help to chop up the data more
