[
https://issues.apache.org/jira/browse/SPARK-13625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-13625:
---------------------------------
Description:
In PySpark params.__init__.py, the method {{Param.params()}} returns a list of
Params belonging to that object. This methos should not check an attribute to
be an instance of {{Param}} if it is a property (uses the {{@property}}
decorator). This causes the property to be invoked to 'get' the attribute, and
that can lead to an error, depending on the property. If an attribute is a
property it will not be an ML {{Param}}, so no need to check it.
I came across this in working on SPARK-13430 while adding
{{LinearRegressionModel.summary}} as a property to give a training summary,
similar to the Scala API. It is possible that a training summary does not
exist and will then raise an exception if the {{summary}} property is invoked.
Calling {{getattr(self, x)}} will cause the property to be invoked if {{x}} is
a property. To fix this, just need to check if it a class property before
making the call to {{getattr()}} in {{Param.params()}}.
was:
In PySpark params.__init__.py, the method {{Param.params()}} returns a list of
Params belonging to that object. This methos should not check an attribute to
be an instance of {{Param}} if it is a property (uses the {{@property}}
decorator). This causes the property to be invoked to 'get' the attribute, and
that can lead to an error, depending on the property. If an attribute is a
property it will not be an ML {{Param}}, so no need to check it.
I came across this in working on SPARK-13430 while adding
{{LinearRegressionModel.summary}} as a property to give a training summary,
similar to the Scala API. It is possible that a training summary does not
exist and will then raise an exception if the {{summary}} property is invoked.
Calling {{getattr(self, x)}} will cause the property to be invoked if {{x}} is
a property. To fix this, just need to check if it a class property before
making the call to {{getattr()}}.
> PySpark-ML method to get list of params for an obj should not check property
> attr
> ---------------------------------------------------------------------------------
>
> Key: SPARK-13625
> URL: https://issues.apache.org/jira/browse/SPARK-13625
> Project: Spark
> Issue Type: Improvement
> Components: ML, PySpark
> Reporter: Bryan Cutler
>
> In PySpark params.__init__.py, the method {{Param.params()}} returns a list
> of Params belonging to that object. This methos should not check an
> attribute to be an instance of {{Param}} if it is a property (uses the
> {{@property}} decorator). This causes the property to be invoked to 'get'
> the attribute, and that can lead to an error, depending on the property. If
> an attribute is a property it will not be an ML {{Param}}, so no need to
> check it.
> I came across this in working on SPARK-13430 while adding
> {{LinearRegressionModel.summary}} as a property to give a training summary,
> similar to the Scala API. It is possible that a training summary does not
> exist and will then raise an exception if the {{summary}} property is
> invoked.
> Calling {{getattr(self, x)}} will cause the property to be invoked if {{x}}
> is a property. To fix this, just need to check if it a class property before
> making the call to {{getattr()}} in {{Param.params()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
