Hi Cesar,
Can you try 1.3.1 (
https://spark.apache.org/releases/spark-release-1-3-1.html) and see if it
still shows the error?
Thanks,
Yin
On Fri, Apr 17, 2015 at 1:58 PM, Reynold Xin r...@databricks.com wrote:
This is strange. cc the dev list since it might be a bug.
On Thu, Apr 16,
I have a data frame in which I load data from a hive table. And my issue is
that the data frame is missing the columns that I need to query.
For example:
val newdataset = dataset.where(dataset(label) === 1)
gives me an error like the following:
ERROR yarn.ApplicationMaster: User class threw
Never mind. I found the solution:
val newDataFrame = hc.createDataFrame(hiveLoadedDataFrame.rdd,
hiveLoadedDataFrame.schema)
which translate to convert the data frame to rdd and back again to data
frame. Not the prettiest solution, but at least it solves my problems.
Thanks,
Cesar Flores
On
