HyukjinKwon commented on code in PR #42118:
URL: https://github.com/apache/spark/pull/42118#discussion_r1275790827
##########
python/pyspark/ml/deepspeed/tests/test_deepspeed_distributor.py:
##########
@@ -164,6 +174,178 @@ def test_create_torchrun_command_distributed(self) ->
None:
self.assertEqual(distributed_cmd_args_expected,
distributed_command_with_args)
+def _create_basic_function() -> Callable:
+ # TODO: swap out with better test function
+ # once Deepspeed better supports CPU
+ def pythagoras(leg1: float, leg2: float) -> float:
+ import deepspeed
+
+ print(deepspeed.__version__)
+ return (leg1 * leg1 + leg2 * leg2) ** 0.5
+
+ return pythagoras
+
+
+@contextmanager
+def _create_pytorch_training_test_file():
+ # TODO: swap with better training file like below:
+ # import deepspeed
Review Comment:
should better move commented codes. We can add them back when we actually
need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
