Re: JdbcRDD Constructor

2015-10-20 Thread satish chandra j
Hi Deenar, Thanks for your valuable inputs Here is a situation, if a Source Table does not have any such column(unique values,numeric and sequential) which is suitable as Partition Column to be specified for JDBCRDD Constructor or DataSource API.How to proceed further on this scenario and also

Re: JdbcRDD Constructor

2015-10-20 Thread Deenar Toraskar
n, if a Source Table does not have any such > column(unique values,numeric and sequential) which is suitable as Partition > Column to be specified for JDBCRDD Constructor or DataSource API.How to > proceed further on this scenario and also let me know if any default > approach Spark

Re: JdbcRDD Constructor

2015-09-24 Thread satish chandra j
gt;>>> Regards, >>>> Satish Chandra >>>> >>>> On Wed, Sep 23, 2015 at 4:02 PM, Rishitesh Mishra < >>>> rishi80.mis...@gmail.com> wrote: >>>> >>>>> Which version of Spark you are using ?? I can get correct results

Re: JdbcRDD Constructor

2015-09-24 Thread Deenar Toraskar
i80.mis...@gmail.com> wrote: >>>> >>>>> I am using Spark 1.5. I always get count = 100, irrespective of num >>>>> partitions. >>>>> >>>>> On Wed, Sep 23, 2015 at 5:00 PM, satish chandra j < >>>>> jsatishchan...@

Re: JdbcRDD Constructor

2015-09-23 Thread Rishitesh Mishra
han...@gmail.com > wrote: > HI All, > > JdbcRDD constructor has following parameters, > > *JdbcRDD > <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/rdd/JdbcRDD.html#JdbcRDD(org.apache.spark.SparkContext,%20scala.Function0,%20java.lang.String,%20long,%20lon

Re: JdbcRDD Constructor

2015-09-23 Thread Rishitesh Mishra
te precisely for this (JdbcRDDSuite) >> . >> I changed according to your input and got correct results from this test >> suite. >> >> On Wed, Sep 23, 2015 at 11:00 AM, satish chandra j < >> jsatishchan...@gmail.com> wrote: >> >>> HI All, >>&

Re: JdbcRDD Constructor

2015-09-23 Thread satish chandra j
dbcRDD. Infact there is a test suite precisely for this (JdbcRDDSuite) >>> . >>> I changed according to your input and got correct results from this test >>> suite. >>> >>> On Wed, Sep 23, 2015 at 11:00 AM, satish chandra j < >>> jsatishchan...@

Re: JdbcRDD Constructor

2015-09-23 Thread satish chandra j
I can get correct results using > JdbcRDD. Infact there is a test suite precisely for this (JdbcRDDSuite) . > I changed according to your input and got correct results from this test > suite. > > On Wed, Sep 23, 2015 at 11:00 AM, satish chandra j < > jsatishchan...@gmail.com>

Re: JdbcRDD Constructor

2015-09-23 Thread Deenar Toraskar
;> On Wed, Sep 23, 2015 at 4:02 PM, Rishitesh Mishra < >>>> rishi80.mis...@gmail.com> wrote: >>>> >>>>> Which version of Spark you are using ?? I can get correct results >>>>> using JdbcRDD. Infact there is a test suite precisely for this

JdbcRDD Constructor

2015-09-22 Thread satish chandra j
HI All, JdbcRDD constructor has following parameters, *JdbcRDD <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/rdd/JdbcRDD.html#JdbcRDD(org.apache.spark.SparkContext, scala.Function0, java.lang.String, long, long, int, scala.Function1, scala.reflect.ClassTag)>*(SparkContext