This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c6ad0b0825d7 [SPARK-40559][PYTHON][FOLLOW-UP] Fix linter for
`getfullargspec`
c6ad0b0825d7 is described below
commit c6ad0b0825d7174a9b24b13156a0fa7c59056158
Author: Hyukjin Kwon <gurwls...@apache.org>
AuthorDate: Mon Dec 4 11:15:39 2023 +0900
[SPARK-40559][PYTHON][FOLLOW-UP] Fix linter for `getfullargspec`
### What changes were proposed in this pull request?
This PR proposes to use `inspect.getfullargspec` instead of unimported
`getfullargspec`. This PR is a followup of
https://github.com/apache/spark/pull/38624.
### Why are the changes needed?
To recover the CI.
It fails as below:
```
./python/pyspark/worker.py:749:19: F821 undefined name 'getfullargspec'
argspec = getfullargspec(chained_func) # signature was lost when
wrapping it
^
./python/pyspark/worker.py:757:19:
F8[21](https://github.com/apache/spark/actions/runs/7080907452/job/19269484124#step:21:22)
undefined name 'getfullargspec'
argspec = getfullargspec(chained_func) # signature was lost when
wrapping it
```
https://github.com/apache/spark/actions/runs/7080907452/job/19269484124
It was caused by the logical conflict w/
https://github.com/apache/spark/commit/f5e4e84ce3a7f65407d07cfc3eed2f51837527c1
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually tested via `linter-python`.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #44141 from HyukjinKwon/SPARK-40559-followup2.
Authored-by: Hyukjin Kwon <gurwls...@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
python/pyspark/worker.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/worker.py b/python/pyspark/worker.py
index 2534238b43cb..158e3ae62bb7 100644
--- a/python/pyspark/worker.py
+++ b/python/pyspark/worker.py
@@ -746,7 +746,7 @@ def read_single_udf(pickleSer, infile, eval_type,
runner_conf, udf_index):
argspec = inspect.getfullargspec(chained_func) # signature was lost
when wrapping it
return args_offsets, wrap_grouped_map_pandas_udf(func, return_type,
argspec, runner_conf)
elif eval_type == PythonEvalType.SQL_GROUPED_MAP_ARROW_UDF:
- argspec = getfullargspec(chained_func) # signature was lost when
wrapping it
+ argspec = inspect.getfullargspec(chained_func) # signature was lost
when wrapping it
return args_offsets, wrap_grouped_map_arrow_udf(func, return_type,
argspec, runner_conf)
elif eval_type == PythonEvalType.SQL_GROUPED_MAP_PANDAS_UDF_WITH_STATE:
return args_offsets, wrap_grouped_map_pandas_udf_with_state(func,
return_type)
@@ -754,7 +754,7 @@ def read_single_udf(pickleSer, infile, eval_type,
runner_conf, udf_index):
argspec = inspect.getfullargspec(chained_func) # signature was lost
when wrapping it
return args_offsets, wrap_cogrouped_map_pandas_udf(func, return_type,
argspec, runner_conf)
elif eval_type == PythonEvalType.SQL_COGROUPED_MAP_ARROW_UDF:
- argspec = getfullargspec(chained_func) # signature was lost when
wrapping it
+ argspec = inspect.getfullargspec(chained_func) # signature was lost
when wrapping it
return args_offsets, wrap_cogrouped_map_arrow_udf(func, return_type,
argspec, runner_conf)
elif eval_type == PythonEvalType.SQL_GROUPED_AGG_PANDAS_UDF:
return wrap_grouped_agg_pandas_udf(func, args_offsets, kwargs_offsets,
return_type)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
