[
https://issues.apache.org/jira/browse/SPARK-38882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-38882.
----------------------------------
Resolution: Fixed
Fixed in https://github.com/apache/spark/pull/36167
> The usage logger attachment logic should handle static methods properly.
> ------------------------------------------------------------------------
>
> Key: SPARK-38882
> URL: https://issues.apache.org/jira/browse/SPARK-38882
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.2.1, 3.3.0
> Reporter: Takuya Ueshin
> Priority: Major
>
> The usage logger attachment logic has an issue when handling static methods.
> For example,
> {code}
> $ PYSPARK_PANDAS_USAGE_LOGGER=pyspark.pandas.usage_logging.usage_logger
> ./bin/pyspark
> {code}
> {code:python}
> >>> import pyspark.pandas as ps
> >>> psdf = ps.DataFrame({"a": [1,2,3], "b": [4,5,6]})
> >>> psdf.from_records([(1, 2), (3, 4)])
> A function `DataFrame.from_records(data, index, exclude, columns,
> coerce_float, nrows)` was failed after 2007.430 ms: 0
> Traceback (most recent call last):
> ...
> {code}
> without usage logger:
> {code:python}
> >>> import pyspark.pandas as ps
> >>> psdf = ps.DataFrame({"a": [1,2,3], "b": [4,5,6]})
> >>> psdf.from_records([(1, 2), (3, 4)])
> 0 1
> 0 1 2
> 1 3 4
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
