Thanks Matei. What does unionAll do if the input RDD schemas are not 100%
compatible. Does it take the union of the columns and generalize the types?
thanks
Daniel
On Sat, Nov 1, 2014 at 6:08 PM, Matei Zaharia <matei.zaha...@gmail.com>
wrote:
> Try unionAll, which is a special method on SchemaRDDs that keeps the
> schema on the results.
>
> Matei
>
> > On Nov 1, 2014, at 3:57 PM, Daniel Mahler <dmah...@gmail.com> wrote:
> >
> > I would like to combine 2 parquet tables I have create.
> > I tried:
> >
> > sc.union(sqx.parquetFile("fileA"), sqx.parquetFile("fileB"))
> >
> > but that just returns RDD[Row].
> > How do I combine them to get a SchemaRDD[Row]?
> >
> > thanks
> > Daniel
>
>
