You can set the dbtable to this: .option("dbtable", "(select * from master_schema where 'TID' = '100_0')")
HTH, Todd On Thu, Jul 21, 2016 at 10:59 AM, sujeet jog <sujeet....@gmail.com> wrote: > I have a table of size 5GB, and want to load selective rows into dataframe > instead of loading the entire table in memory, > > > For me memory is a constraint hence , and i would like to peridically load > few set of rows and perform dataframe operations on it, > > , > for the "dbtable" is there a way to perform select * from master_schema > where 'TID' = '100_0'; > which can load only this to memory as dataframe . > > > > Currently I'm using code as below > val df = sqlContext.read .format("jdbc") > .option("url", url) > .option("dbtable", "master_schema").load() > > > Thansk, > Sujeet >