Well I don't know about postgres but you can limit the number of columns
abd rows fetched via JDBC at source rather than loading and filtering them
in Spark
val c = HiveContext.load("jdbc",
Map("url" -> _ORACLEserver,
"dbtable" -> "(SELECT to_char(CHANNEL_ID) AS CHANNEL_ID, CHANNEL_DESC FROM
I try to load some rows from a big SQL table. Here is my code:
===
jdbcDF = sqlContext.read.format("jdbc").options(
url="jdbc:postgresql://...",
dbtable="mytable",
partitionColumn="t",
lowerBound=1451577600,
upperBound=1454256000,
numPartitions=1).load()
