I found my problem. I was calling setParameterValue(defaultValue) more than
one time in the hierarchy of my classes.
Thanks!
On Mon, Feb 15, 2016 at 6:34 PM, Cesar Flores <ces...@gmail.com> wrote:
>
> I have a set of transformers (each with specific parameters) in spark
> 1.3.1. I have two versions, one that works and one that does not:
>
> 1.- working version
> //featureprovidertransformer contains already a set of ml params
> class DemographicTransformer(override val uid: String) extends
> FeatureProviderTransformer {
>
> def this() = this(Identifiable.randomUID("demo-transformer"))
> override def copy(extra: ParamMap): DemographicTransformer =
> defaultCopy(extra)
>
> ....
>
> }
>
> 2.- not working version
> class DemographicTransformer(override val uid: String) extends
> FeatureProviderTransformer {
>
> def this() = this(Identifiable.randomUID("demo-transformer"))
> override def copy(extra: ParamMap): DemographicTransformer =
> defaultCopy(extra)
>
> *//add another transformer parameter*
> final val anotherParam: Param[String] = new Param[String](this,
> "anotherParam", "dummy parameter")
> ....
>
> }
>
> Somehow adding an *anotherParam* to my class make it fail, with the
> following error:
>
> [info] java.lang.NullPointerException:
> [info] at
> org.apache.spark.ml.param.Params$$anonfun$hasParam$1.apply(params.scala:408)
> [info] at
> org.apache.spark.ml.param.Params$$anonfun$hasParam$1.apply(params.scala:408)
> [info] at
> scala.collection.IndexedSeqOptimized$$anonfun$exists$1.apply(IndexedSeqOptimized.scala:40)
> [info] at
> scala.collection.IndexedSeqOptimized$$anonfun$exists$1.apply(IndexedSeqOptimized.scala:40)
> [info] at
> scala.collection.IndexedSeqOptimized$class.segmentLength(IndexedSeqOptimized.scala:189)
> [info] at
> scala.collection.mutable.ArrayOps$ofRef.segmentLength(ArrayOps.scala:108)
> [info] at
> scala.collection.GenSeqLike$class.prefixLength(GenSeqLike.scala:92)
> [info] at
> scala.collection.mutable.ArrayOps$ofRef.prefixLength(ArrayOps.scala:108)
> [info] at
> scala.collection.IndexedSeqOptimized$class.exists(IndexedSeqOptimized.scala:40)
> [info] at
> scala.collection.mutable.ArrayOps$ofRef.exists(ArrayOps.scala:108)
>
> Debugging the params.scala class shows me that actually adding
> *anotherParam* *replace all parameters by a single one called allParams.*
>
> *Does anyone have any idea of what I may be doing wrong. My guess is that
> I am doing something weird in my class hierarchy but can not figure out
> what.*
>
>
> Thanks!
> --
> Cesar Flores
>
--
Cesar Flores
