Eyal Allweil created DATAFU-159:
-----------------------------------
Summary: Add diff functionality to datafu-spark
Key: DATAFU-159
URL: https://issues.apache.org/jira/browse/DATAFU-159
Project: DataFu
Issue Type: New Feature
Reporter: Eyal Allweil
A useful feature when examining results is the ability to clearly understand
the differences between two datasets - for example, doing regressions between
expected and actual results.
Spark provides the _except_ functionality, but this is often not enough for
this - for example, see [this question on Stack
Overflow.|https://stackoverflow.com/questions/44338412/how-to-compare-two-dataframe-and-print-columns-that-are-different-in-scala]
Datafu-pig had a macro for doing this, and this could be a useful addition to
datafu-spark.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
