Hi Chetan, You can substract the data frame or use except operation. First DF contains full rows. Second DF contains unique rows (post remove duplicates) Subtract first and second DF .
hope this helps Thanks Sachit On Tue, Jun 22, 2021, 22:23 Chetan Khatri <chetan.opensou...@gmail.com> wrote: > Hi Spark Users, > > I want to use DropDuplicate, but those records which I discard. I > would like to log to the instrumental table. > > What would be the best approach to do that? > > Thanks >
