Re: pyspark unable to convert dataframe column to a vector: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Hi Jeff Sorry I did not respond sooner. I was out of town Here is the code I use to initialize the HiveContext # load data set from pyspark.sql import HiveContext #,SQLContext, Row # window functions require HiveContext (spark 2.x will not require hive) #sqlContext = SQLContext(sc)

Re: pyspark unable to convert dataframe column to a vector: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

According the stack trace, it seems the HiveContext is not initialized correctly. Do you have any more error message ? On Tue, Mar 29, 2016 at 9:29 AM, Andy Davidson < a...@santacruzintegration.com> wrote: > I am using pyspark spark-1.6.1-bin-hadoop2.6 and python3. I have a data > frame with a

pyspark unable to convert dataframe column to a vector: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

I am using pyspark spark-1.6.1-bin-hadoop2.6 and python3. I have a data frame with a column I need to convert to a sparse vector. I get an exception Any idea what my bug is? Kind regards Andy Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext. :