HI All,
In addition to Case Class limitation in Scala, I finding Tuple limitation
too please find the explanation below
//Query to pull data from Source Table
var SQL_RDD= new JdbcRDD( sc, ()=>
DriverManager.getConnection(url,user,pass),"select col1, col2,
col3......col 37 from schema.Table LIMIT ? OFFSET ?",100,0,*1*,(r:
ResultSet) => (r.getInt("col1"),r.getInt("col2").......r.getInt("col37")))
//Define Case Class
case class sqlrow(col1:Int,col2:Int......col37)
var SchSQL= SQL_RDD.map(p => new sqlrow(p._1,p._2.....p._37))
followed by apply CreateSchema to RDD and than apply registerTempTable for
defining a table to make use in SQL Context in Spark
As per the above SQL query I need to fetch 37 columns from the source
table, but it seems Scala has tuple restriction which I am defining by r
ResultSet variable in the above SQL, please let me know if any work around
for the same
Regards,
Satish Chandra
On Thu, Sep 24, 2015 at 3:18 PM, satish chandra j <jsatishchan...@gmail.com>
wrote:
> HI All,
> As it is for SQL purpose I understand, need to go ahead with Custom Case
> Class approach
> Could anybody have a sample code for creating Custom Case Class to refer
> which would be really helpful
>
> Regards,
> Satish Chandra
>
> On Thu, Sep 24, 2015 at 2:51 PM, Adrian Tanase <atan...@adobe.com> wrote:
>
>> +1 on grouping the case classes and creating a hierarchy – as long as you
>> use the data programatically. For DataFrames / SQL the other ideas probably
>> scale better…
>>
>> From: Ted Yu
>> Date: Wednesday, September 23, 2015 at 7:07 AM
>> To: satish chandra j
>> Cc: user
>> Subject: Re: Scala Limitation - Case Class definition with more than 22
>> arguments
>>
>> Can you switch to 2.11 ?
>>
>> The following has been fixed in 2.11:
>> https://issues.scala-lang.org/browse/SI-7296
>>
>> Otherwise consider packaging related values into a case class of their
>> own.
>>
>> On Tue, Sep 22, 2015 at 8:48 PM, satish chandra j <
>> jsatishchan...@gmail.com> wrote:
>>
>>> HI All,
>>> Do we have any alternative solutions in Scala to avoid limitation in
>>> defining a Case Class having more than 22 arguments
>>>
>>> We are using Scala version 2.10.2, currently I need to define a case
>>> class with 37 arguments but getting an error as "*error:Implementation
>>> restriction:caseclasses cannot have more than 22parameters.*"
>>>
>>> It would be a great help if any inputs on the same
>>>
>>> Regards,
>>> Satish Chandra
>>>
>>>
>>>
>>
>
