Re: Suggestion on Join Approach with Spark

2019-05-15 Thread Chetan Khatri
Hello Nicholas, I sincerely apologise. Thanks On Wed, May 15, 2019 at 11:34 PM Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > This kind of question is for the User list, or for something like Stack > Overflow. It's not on topic here. > > The dev list (i.e. this list) is for

Re: Suggestion on Join Approach with Spark

2019-05-15 Thread Nicholas Chammas
This kind of question is for the User list, or for something like Stack Overflow. It's not on topic here. The dev list (i.e. this list) is for discussions about the development of Spark itself. On Wed, May 15, 2019 at 1:50 PM Chetan Khatri wrote: > Any one help me, I am confused. :( > > On

Re: Suggestion on Join Approach with Spark

2019-05-15 Thread Chetan Khatri
Any one help me, I am confused. :( On Wed, May 15, 2019 at 7:28 PM Chetan Khatri wrote: > Hello Spark Developers, > > I have a question on Spark Join I am doing. > > I have a full load data from RDBMS and storing at HDFS let's say, > > val historyDF =

Suggestion on Join Approach with Spark

2019-05-15 Thread Chetan Khatri
Hello Spark Developers, I have a question on Spark Join I am doing. I have a full load data from RDBMS and storing at HDFS let's say, val historyDF = spark.read.parquet(*"/home/test/transaction-line-item"*) and I am getting changed data at seperate hdfs path,let's say; val deltaDF =