Re: Work out date column in CSV more than 6 months old (datediff or something)

2016-03-22 Thread James Hammerton
tps://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > On 22 March 2016 at 00:26, Silvio Fiorito <silvio.fior...@granturing.com> > wrote: > >> There’s a months_between funct

Re: Work out date column in CSV more than 6 months old (datediff or something)

2016-03-22 Thread Mich Talebzadeh
rrent_date, $”Payment Date”) > 6).show > > From: Mich Talebzadeh <mich.talebza...@gmail.com> > Date: Monday, March 21, 2016 at 5:53 PM > To: "user @spark" <user@spark.apache.org> > Subject: Work out date column in CSV more than 6 months old (datediff or > somet

Re: Work out date column in CSV more than 6 months old (datediff or something)

2016-03-21 Thread Silvio Fiorito
t; <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Work out date column in CSV more than 6 months old (datediff or something) Hi, For test purposes I am reading in a simple csv file as follows: val df = sqlContext.read.format("com.databricks.spark.csv").option

Work out date column in CSV more than 6 months old (datediff or something)

2016-03-21 Thread Mich Talebzadeh
Hi, For test purposes I am reading in a simple csv file as follows: val df = sqlContext.read.format("com.databricks.spark.csv").option("inferSchema", "true").option("header", "true").load("/data/stg/table2") df: org.apache.spark.sql.DataFrame = [Invoice Number: string, Payment date: string, Net: