Re: DataFrame defined within conditional IF ELSE statement

2016-09-18 Thread Silvio Fiorito
REATED,'SECURITY,'PRICE) case _ => sys.err(“no valid option provided”) } df.printSchema() Thanks, Silvio From: Mich Talebzadeh <mich.talebza...@gmail.com<mailto:mich.talebza...@gmail.com>> Date: Saturday, September 17, 2016 at 4:18 PM To: "user @spark" <user@spark.ap

Re: DataFrame defined within conditional IF ELSE statement

2016-09-18 Thread Mich Talebzadeh
("test.marketData").select('TIMECREATED,' > SECURITY,'PRICE) > > case 3 => spark.table("test.marketDataParquet").select(' > TIMECREATED,'SECURITY,'PRICE) > > case _ => sys.err(“no valid option provided”) > > } > > > > df.printSchema(

Re: DataFrame defined within conditional IF ELSE statement

2016-09-18 Thread Silvio Fiorito
ase _ => sys.err(“no valid option provided”) } df.printSchema() Thanks, Silvio From: Mich Talebzadeh <mich.talebza...@gmail.com> Date: Saturday, September 17, 2016 at 4:18 PM To: "user @spark" <user@spark.apache.org> Subject: DataFrame defined within conditional IF ELSE sta

Re: DataFrame defined within conditional IF ELSE statement

2016-09-18 Thread Mich Talebzadeh
any opinion on this please? Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your

DataFrame defined within conditional IF ELSE statement

2016-09-17 Thread Mich Talebzadeh
In Spark 2 this gives me an error in a conditional IF ELSE statement I recall seeing the same in standard SQL I am doing a test for different sources (text file, ORC or Parquet) to be read in dependent on value of var option I wrote this import org.apache.spark.sql.functions._ import