[ https://issues.apache.org/jira/browse/SPARK-13625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Cutler updated SPARK-13625: --------------------------------- Description: In PySpark params.__init__.py, the method {{Param.params()}} returns a list of Params belonging to that object. This method should not check an attribute to be an instance of {{Param}} if it is a property (uses the {{@property}} decorator). This causes the property to be invoked to 'get' the attribute, and that can lead to an error, depending on the property. If an attribute is a property it will not be an ML {{Param}}, so no need to check it. I came across this in working on SPARK-13430 while adding {{LinearRegressionModel.summary}} as a property to give a training summary, similar to the Scala API. It is possible that a training summary does not exist and will then raise an exception if the {{summary}} property is invoked. Calling {{getattr(self, x)}} will cause the property to be invoked if {{x}} is a property. To fix this, just need to check if it a class property before making the call to {{getattr()}} in {{Param.params()}}. was: In PySpark params.__init__.py, the method {{Param.params()}} returns a list of Params belonging to that object. This methos should not check an attribute to be an instance of {{Param}} if it is a property (uses the {{@property}} decorator). This causes the property to be invoked to 'get' the attribute, and that can lead to an error, depending on the property. If an attribute is a property it will not be an ML {{Param}}, so no need to check it. I came across this in working on SPARK-13430 while adding {{LinearRegressionModel.summary}} as a property to give a training summary, similar to the Scala API. It is possible that a training summary does not exist and will then raise an exception if the {{summary}} property is invoked. Calling {{getattr(self, x)}} will cause the property to be invoked if {{x}} is a property. To fix this, just need to check if it a class property before making the call to {{getattr()}} in {{Param.params()}}. > PySpark-ML method to get list of params for an obj should not check property > attr > --------------------------------------------------------------------------------- > > Key: SPARK-13625 > URL: https://issues.apache.org/jira/browse/SPARK-13625 > Project: Spark > Issue Type: Improvement > Components: ML, PySpark > Reporter: Bryan Cutler > > In PySpark params.__init__.py, the method {{Param.params()}} returns a list > of Params belonging to that object. This method should not check an > attribute to be an instance of {{Param}} if it is a property (uses the > {{@property}} decorator). This causes the property to be invoked to 'get' > the attribute, and that can lead to an error, depending on the property. If > an attribute is a property it will not be an ML {{Param}}, so no need to > check it. > I came across this in working on SPARK-13430 while adding > {{LinearRegressionModel.summary}} as a property to give a training summary, > similar to the Scala API. It is possible that a training summary does not > exist and will then raise an exception if the {{summary}} property is > invoked. > Calling {{getattr(self, x)}} will cause the property to be invoked if {{x}} > is a property. To fix this, just need to check if it a class property before > making the call to {{getattr()}} in {{Param.params()}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org