Thanks Todd. On Thu, Jul 21, 2016 at 9:18 PM, Todd Nist <tsind...@gmail.com> wrote:
> You can set the dbtable to this: > > .option("dbtable", "(select * from master_schema where 'TID' = '100_0')") > > HTH, > > Todd > > > On Thu, Jul 21, 2016 at 10:59 AM, sujeet jog <sujeet....@gmail.com> wrote: > >> I have a table of size 5GB, and want to load selective rows into >> dataframe instead of loading the entire table in memory, >> >> >> For me memory is a constraint hence , and i would like to peridically >> load few set of rows and perform dataframe operations on it, >> >> , >> for the "dbtable" is there a way to perform select * from master_schema >> where 'TID' = '100_0'; >> which can load only this to memory as dataframe . >> >> >> >> Currently I'm using code as below >> val df = sqlContext.read .format("jdbc") >> .option("url", url) >> .option("dbtable", "master_schema").load() >> >> >> Thansk, >> Sujeet >> > >