Re: Merging two avro RDD/DataFrames

under a minute with the correct resources… -adrian From: TEST ONE Date: Tuesday, September 29, 2015 at 3:00 AM To: "user@spark.apache.org<mailto:user@spark.apache.org>" Subject: Merging two avro RDD/DataFrames I have a daily update of modified users (~100s) output as avro fro

Merging two avro RDD/DataFrames

I have a daily update of modified users (~100s) output as avro from ETL. I’d need to find and merge with existing corresponding members in a master avro file (~100,000s) The merge operation involves merging a ‘profiles’ Map between the matching records. What would be the