Schema merging is not the feature you are looking for. It is designed when you are adding new records (that are not associated with old records), which may or may not have new or missing columns.
In your case it looks like you have two datasets that you want to load separately and join on a key. On Wed, Apr 15, 2015 at 5:59 AM, Jaonary Rabarisoa <jaon...@gmail.com> wrote: > Hi all, > > If you follow the example of schema merging in the spark documentation > http://spark.apache.org/docs/latest/sql-programming-guide.html#schema-merging > you obtain the following results when you want to load the result data : > > single triple double > 1 3 null > 2 6 null > 4 12 null > 3 9 null > 5 15 null > 1 null 2 > 2 null 4 > 4 null 8 > 3 null 6 > 5 null 10 > > How to remove these null value and get something more logical like : > > single triple double > 1 3 2 > 2 6 4 > 4 12 8 > 3 9 6 > 5 15 10 > > Bests, > > Jao >
