Re: SparkSession for RDBMS

[Takeshi Yamamuro]

Hi,

If these bounaries are not given, spark tries to read all the data as a
single parition.
See:
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala#L56

// maropu


On Wed, Aug 3, 2016 at 11:19 PM, Selvam Raman  wrote:

> Hi All,
>
> I would like to read the data from RDBMS to spark (2.0) using
> sparksession. How can i decide upper boundary, lower boundary and
> partitions.
> is there any specific approach available.
>
> How Sqoop2 decides number of partitions, upper and lower boundary if we
> are not specifying anything.
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>



-- 
---
Takeshi Yamamuro

SparkSession for RDBMS

[Selvam Raman]

Hi All,

I would like to read the data from RDBMS to spark (2.0) using sparksession.
How can i decide upper boundary, lower boundary and partitions.
is there any specific approach available.

How Sqoop2 decides number of partitions, upper and lower boundary if we are
not specifying anything.

-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"