Hi, I am working with Spark 1.6 with scala and using Dataframe API . I have a use case where I need to compare two rows and add entry in the new column based on the lookup table for example : My DF looks like : col1 col2 newCol1 street1 person1 street2 person1 area1 street3 person1 area3 street5 person2 street6 person2 area5 street7 person4 street9 person4 area7
loop up table looks like street1 -> street2 - area1 street2 -> street 3 - area3 street5 -> street6 - area5 street 7-> street 9 - area 7 if person moving from street 1 to street 2 then he is reaching area 1 Would really appreciate the help. Thanks, Divya
