jcw024 commented on code in PR #51687:
URL: https://github.com/apache/spark/pull/51687#discussion_r2240698364
##########
python/pyspark/ml/linalg/__init__.py:
##########
@@ -433,7 +433,7 @@ def squared_distance(self, other: Iterable[float]) ->
np.float64:
if isinstance(other, SparseVector):
return other.squared_distance(self)
elif _have_scipy and scipy.sparse.issparse(other):
- return _convert_to_vector(other).squared_distance(self) # type:
ignore[attr-defined]
+ return _convert_to_vector(other).squared_distance(self) # type:
ignore
Review Comment:
is this deletion necessary for the black upgrade? I'm not sure if we want to
remove information from the comment if it's not required by black
##########
dev/requirements.txt:
##########
@@ -76,7 +76,7 @@ graphviz==0.20.3
flameprof==0.4
# TorchDistributor dependencies
-torch<2.6.0
+torch
Review Comment:
is this deletion necessary? If so, have you tested if newer versions of
torch will work?
this pin was added in this PR a few months ago:
https://github.com/apache/spark/pull/49863
##########
python/pyspark/sql/tests/connect/test_parity_python_datasource.py:
##########
@@ -18,8 +18,9 @@
from pyspark.testing.connectutils import ReusedConnectTestCase
-class PythonDataSourceParityTests(BasePythonDataSourceTestsMixin,
ReusedConnectTestCase):
- ...
+class PythonDataSourceParityTests(
+ BasePythonDataSourceTestsMixin, ReusedConnectTestCase
+): ... # noqa: E701
Review Comment:
could you explain why noqa is used here for E701 while tox.ini adds E704 to
the ignore list? Just trying to understand which rules are being excluded where
and for what reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
