Re: Dataframe joins - UnsupportedOperationException: Unimplemented type: IntegerType

2018-07-09 Thread Vadim Semenov
That usually happens when you have different types for a column in some parquet files. In this case, I think you have a column of `Long` type that got a file with `Integer` type, I had to deal with similar problem once. You would have to cast it yourself to Long. On Mon, Jul 9, 2018 at 2:53 PM

Dataframe joins - UnsupportedOperationException: Unimplemented type: IntegerType

2018-07-09 Thread Nirav Patel
I am getting following error after performing joins between 2 dataframe. It happens on call to .show() method. I assume it's an issue with incompatible type but it's been really hard to identify which column of which dataframe have that incompatibility. Any pointers? 11:06:10.304 13700 [Executor