I am writing an UDAF to be applied to a data frame column of type Vector (spark.ml.linalg.Vector). I rely on spark/ml/linalg so that I do not have to go back and forth between dataframe and RDD.
Inside the UDAF, I have to specify a data type for the input, buffer, and output (as usual). VectorUDT is what I would use with spark.mllib.linalg.Vector: https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala However, when I try to import it from spark.ml instead: import org.apache.spark.ml.linalg.VectorUDT I get a runtime error (no errors during the build): class VectorUDT in package linalg cannot be accessed in package org.apache.spark.ml.linalg Is it expected/can you suggest a workaround? I am using Spark 2.0.0 Thanks, Alexey -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/VectorUDT-with-spark-ml-linalg-Vector-tp27542.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
