[GitHub] [spark] SparkQA commented on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

SparkQA commented on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974793626 **[Test build #145481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145481/testReport)** for PR 34610 at commit [`6df9efd`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

SparkQA removed a comment on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974760014 **[Test build #145481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145481/testReport)** for PR 34610 at commit [`6df9efd`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

AmplabJenkins commented on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974793916 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145481/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

AmplabJenkins removed a comment on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974793916 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145481/ -

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974794216 **[Test build #145482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145482/testReport)** for PR 34611 at commit [`705f406`](https://github.com

[GitHub] [spark] zero323 commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

zero323 commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753783981 ## File path: python/pyspark/mllib/linalg/__init__.pyi ## @@ -68,6 +68,7 @@ class Vector: __UDT__: VectorUDT def toArray(self) -> ndarray: ...

[GitHub] [spark] zero323 commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

zero323 commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753784172 ## File path: python/pyspark/mllib/stat/_statistics.py ## @@ -170,10 +190,29 @@ def corr(x, y=None, method=None): if not y: return c

[GitHub] [spark] zero323 commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

zero323 commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753784312 ## File path: python/pyspark/mllib/stat/_statistics.py ## @@ -170,10 +190,29 @@ def corr(x, y=None, method=None): if not y: return c

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753785456 ## File path: python/pyspark/broadcast.py ## @@ -42,7 +43,10 @@ def _from_id(bid): return _broadcastRegistry[bid] -class Broadcast(object): +T =

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753785413 ## File path: python/pyspark/accumulators.py ## @@ -46,7 +48,10 @@ def _deserialize_accumulator(aid, zero_value, accum_param): return accum -

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974799562 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49954/ -- This is an automated message from the Apache

[GitHub] [spark] zero323 commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

zero323 commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753786672 ## File path: python/pyspark/mllib/stat/_statistics.py ## @@ -16,51 +16,57 @@ # import sys +from typing import overload, List, Optional, Union +from t

[GitHub] [spark] zero323 commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

zero323 commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753786802 ## File path: python/pyspark/mllib/stat/_statistics.py ## @@ -16,51 +16,57 @@ # import sys +from typing import overload, List, Optional, Union +from t

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974801850 **[Test build #145483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145483/testReport)** for PR 34611 at commit [`fc67e88`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974806525 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49954/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974808774 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49955/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974811445 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49954/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974811445 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49954/

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974818079 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49955/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974820919 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49955/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974820919 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49955/

[GitHub] [spark] SparkQA commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

SparkQA commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974821403 **[Test build #145484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145484/testReport)** for PR 34374 at commit [`d8561e1`](https://github.com

[GitHub] [spark] zero323 commented on a change in pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

zero323 commented on a change in pull request #34374: URL: https://github.com/apache/spark/pull/34374#discussion_r753803461 ## File path: python/pyspark/rdd.pyi ## @@ -60,6 +60,7 @@ from pyspark.sql._typing import RowLike from py4j.java_gateway import JavaObject # type: ignor

[GitHub] [spark] SparkQA commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

SparkQA commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974825468 **[Test build #145484 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145484/testReport)** for PR 34374 at commit [`d8561e1`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

SparkQA removed a comment on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974821403 **[Test build #145484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145484/testReport)** for PR 34374 at commit [`d8561e1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

SparkQA commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974827753 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49956/ -- This is an automated message from the Apache

[GitHub] [spark] zero323 commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

zero323 commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753808683 ## File path: python/pyspark/mllib/linalg/__init__.pyi ## @@ -68,6 +68,7 @@ class Vector: __UDT__: VectorUDT def toArray(self) -> ndarray: ...

[GitHub] [spark] AmplabJenkins commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

AmplabJenkins commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974830270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145484/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

AmplabJenkins removed a comment on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974830270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145484/ -

[GitHub] [spark] zero323 closed pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

zero323 closed pull request #34374: URL: https://github.com/apache/spark/pull/34374 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] zero323 commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

zero323 commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974837457 Merged to master. Thanks all! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

SparkQA commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974838415 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49956/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974839784 **[Test build #145482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145482/testReport)** for PR 34611 at commit [`705f406`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974794216 **[Test build #145482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145482/testReport)** for PR 34611 at commit [`705f406`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974841693 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145482/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

AmplabJenkins commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974841691 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49956/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

AmplabJenkins removed a comment on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974841691 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49956/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974841693 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145482/ -

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753819607 ## File path: python/pyspark/context.py ## @@ -125,30 +144,36 @@ class SparkContext(object): ValueError: ... """ -_gateway = None -_jv

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753819793 ## File path: python/pyspark/context.py ## @@ -180,20 +205,20 @@ def __init__( def _do_init( self, -master, -appName, -

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753820635 ## File path: python/pyspark/context.py ## @@ -221,7 +246,7 @@ def _do_init( if environment: for key, value in environment.items():

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753822078 ## File path: python/pyspark/context.py ## @@ -244,26 +269,32 @@ def _do_init( self.environment["PYTHONHASHSEED"] = os.environ.get("PYTHONHASHSEE

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753822125 ## File path: python/pyspark/context.py ## @@ -244,26 +269,32 @@ def _do_init( self.environment["PYTHONHASHSEED"] = os.environ.get("PYTHONHASHSEE

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753822162 ## File path: python/pyspark/context.py ## @@ -244,26 +269,32 @@ def _do_init( self.environment["PYTHONHASHSEED"] = os.environ.get("PYTHONHASHSEE

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753823073 ## File path: python/pyspark/context.py ## @@ -244,26 +269,32 @@ def _do_init( self.environment["PYTHONHASHSEED"] = os.environ.get("PYTHONHASHSEE

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753823212 ## File path: python/pyspark/context.py ## @@ -244,26 +269,32 @@ def _do_init( self.environment["PYTHONHASHSEED"] = os.environ.get("PYTHONHASHSEE

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753823738 ## File path: python/pyspark/context.py ## @@ -324,21 +359,23 @@ def _do_init( self.profiler_collector = None # create a signal ha

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753823799 ## File path: python/pyspark/context.py ## @@ -324,21 +359,23 @@ def _do_init( self.profiler_collector = None # create a signal ha

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974849693 **[Test build #145483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145483/testReport)** for PR 34611 at commit [`fc67e88`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974801850 **[Test build #145483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145483/testReport)** for PR 34611 at commit [`fc67e88`](https://gi

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753824526 ## File path: python/pyspark/context.py ## @@ -324,21 +359,23 @@ def _do_init( self.profiler_collector = None # create a signal ha

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753823073 ## File path: python/pyspark/context.py ## @@ -244,26 +269,32 @@ def _do_init( self.environment["PYTHONHASHSEED"] = os.environ.get("PYTHONHASHSEE

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753825024 ## File path: python/pyspark/context.py ## @@ -434,17 +481,17 @@ def getOrCreate(cls, conf=None): with SparkContext._lock: if SparkC

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974853016 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145483/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974853016 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145483/ -

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753827160 ## File path: python/pyspark/context.py ## @@ -565,9 +614,9 @@ def range(self, start, end=None, step=1, numSlices=None): end = start

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753827160 ## File path: python/pyspark/context.py ## @@ -565,9 +614,9 @@ def range(self, start, end=None, step=1, numSlices=None): end = start

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753827648 ## File path: python/pyspark/context.py ## @@ -587,10 +636,10 @@ def parallelize(self, c, numSlices=None): step = c[1] - c[0] if size > 1 els

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753829058 ## File path: python/pyspark/context.py ## @@ -609,16 +658,20 @@ def f(split, iterator): batchSize = max(1, min(len(c) // numSlices, self._batchS

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753829321 ## File path: python/pyspark/context.py ## @@ -787,8 +844,8 @@ def binaryRecords(self, path, recordLength): """ return RDD(self._jsc.bin

[GitHub] [spark] srowen commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/ja

srowen commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-974858343 Jenkins test this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [spark] sarutak commented on pull request #34628: [SPARK-37354][K8S][TESTS] Make the Java version installed on the container image used by the K8s integration tests with SBT configurable

sarutak commented on pull request #34628: URL: https://github.com/apache/spark/pull/34628#issuecomment-974859464 I'll merge this as I have one LGTM from @holdenk . But I'll rebase before merge, just in case. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #34628: [SPARK-37354][K8S][TESTS] Make the Java version installed on the container image used by the K8s integration tests with SBT configurable

SparkQA commented on pull request #34628: URL: https://github.com/apache/spark/pull/34628#issuecomment-974863541 **[Test build #145485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145485/testReport)** for PR 34628 at commit [`9b03228`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/j

SparkQA commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-974863560 **[Test build #145486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145486/testReport)** for PR 34620 at commit [`e6c469e`](https://github.com

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753833289 ## File path: python/pyspark/ml/clustering.py ## @@ -17,6 +17,7 @@ import sys import warnings +from typing import cast Review comment: `pyspar

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753833390 ## File path: python/pyspark/pandas/spark/functions.py ## @@ -40,7 +40,7 @@ def repeat(col: Column, n: Union[int, Column]) -> Column: """ Repeat

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753833435 ## File path: python/pyspark/rdd.py ## @@ -250,7 +251,10 @@ def __call__(self, k): return self.partitionFunc(k) % self.numPartitions -class R

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753833667 ## File path: python/pyspark/sql/avro/functions.py ## @@ -73,7 +73,7 @@ def from_avro( [Row(value=Row(avro=Row(age=2, name='Alice')))] """ -

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753833929 ## File path: python/pyspark/sql/context.py ## @@ -101,6 +101,8 @@ class SQLContext(object): """ _instantiatedContext: ClassVar[Optional["SQLC

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753834047 ## File path: python/pyspark/sql/pandas/conversion.py ## @@ -583,7 +583,7 @@ def create_RDD_server(): jrdd = self._sc._serialize_to_jvm( # type:

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753834093 ## File path: python/pyspark/sql/session.py ## @@ -137,6 +137,9 @@ class SparkSession(SparkConversionMixin): [(1, 'string', 1.0, 1, True, datetime.da

[GitHub] [spark] SparkQA commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/j

SparkQA commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-974867346 **[Test build #145486 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145486/testReport)** for PR 34620 at commit [`e6c469e`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/sc

SparkQA removed a comment on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-974863560 **[Test build #145486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145486/testReport)** for PR 34620 at commit [`e6c469e`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34628: [SPARK-37354][K8S][TESTS] Make the Java version installed on the container image used by the K8s integration tests with SBT configurable

SparkQA commented on pull request #34628: URL: https://github.com/apache/spark/pull/34628#issuecomment-974869669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49957/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/j

SparkQA commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-974869689 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49958/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scal

AmplabJenkins commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-974872074 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145486/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/tar

AmplabJenkins removed a comment on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-974872074 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145486/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

AmplabJenkins removed a comment on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-974755306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] SparkQA removed a comment on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

SparkQA removed a comment on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974760014 **[Test build #145481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145481/testReport)** for PR 34610 at commit [`6df9efd`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #34674: [SPARK-37419][PYTHON][ML] Rewrite _shared_params_code_gen.py to inline type hints for ml/param/shared.py

AmplabJenkins commented on pull request #34674: URL: https://github.com/apache/spark/pull/34674#issuecomment-974652358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34673: [WIP][SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

AmplabJenkins removed a comment on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-974610792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] dchvn commented on pull request #34673: [WIP][SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

dchvn commented on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-974602693 CC @huaxingao FYI. Please take a look when you find some time, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [spark] AmplabJenkins commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

AmplabJenkins commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974830270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] github-actions[bot] closed pull request #33662: [SPARK-36162][SQL] Support estimation of equal null safe join

github-actions[bot] closed pull request #33662: URL: https://github.com/apache/spark/pull/33662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

[GitHub] [spark] AmplabJenkins commented on pull request #34070: [SPARK-36840][SQL] Support DPP if there is no selective predicate on the filtering side

AmplabJenkins commented on pull request #34070: URL: https://github.com/apache/spark/pull/34070#issuecomment-974593431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] zero323 commented on pull request #34669: [SPARK-37390][PYTHON][DOCS] Add default value to getattr(app, add_javascript)

zero323 commented on pull request #34669: URL: https://github.com/apache/spark/pull/34669#issuecomment-974630664 Merged into master, branch-3.2, branch-3.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-974811445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] SparkQA commented on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

SparkQA commented on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974760014 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[GitHub] [spark] SparkQA removed a comment on pull request #34070: [SPARK-36840][SQL] Support DPP if there is no selective predicate on the filtering side

SparkQA removed a comment on pull request #34070: URL: https://github.com/apache/spark/pull/34070#issuecomment-974582054 **[Test build #145473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145473/testReport)** for PR 34070 at commit [`50484e2`](https://gi

[GitHub] [spark] zero323 commented on a change in pull request #34466: [SPARK-37152][PYTHON] Inline type hints for python/pyspark/context.py

zero323 commented on a change in pull request #34466: URL: https://github.com/apache/spark/pull/34466#discussion_r753785413 ## File path: python/pyspark/accumulators.py ## @@ -46,7 +48,10 @@ def _deserialize_accumulator(aid, zero_value, accum_param): return accum -

[GitHub] [spark] SparkQA removed a comment on pull request #34673: [WIP][SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

SparkQA removed a comment on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-974599139 **[Test build #145475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145475/testReport)** for PR 34673 at commit [`baae935`](https://gi

[GitHub] [spark] zero323 commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

zero323 commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974837457 Merged to master. Thanks all! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

AmplabJenkins commented on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974770281 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] LuciferYang commented on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

LuciferYang commented on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-974644902 thanks all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [spark] SparkQA commented on pull request #34374: [SPARK-37104][PYTHON] Make RDD and DStream covariant

SparkQA commented on pull request #34374: URL: https://github.com/apache/spark/pull/34374#issuecomment-974821403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[GitHub] [spark] HyukjinKwon commented on pull request #34669: [SPARK-37390][PYTHON][DOCS] Add default value to getattr(app, add_javascript)

HyukjinKwon commented on pull request #34669: URL: https://github.com/apache/spark/pull/34669#issuecomment-974734313 Lgtm thx! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [spark] zero323 closed pull request #34669: [SPARK-37390][PYTHON][DOCS] Add default value to getattr(app, add_javascript)

zero323 closed pull request #34669: URL: https://github.com/apache/spark/pull/34669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

AmplabJenkins removed a comment on pull request #34610: URL: https://github.com/apache/spark/pull/34610#issuecomment-974770281 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34070: [SPARK-36840][SQL] Support DPP if there is no selective predicate on the filtering side

AmplabJenkins removed a comment on pull request #34070: URL: https://github.com/apache/spark/pull/34070#issuecomment-974593431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] SparkQA commented on pull request #34628: [SPARK-37354][K8S][TESTS] Make the Java version installed on the container image used by the K8s integration tests with SBT configurable

SparkQA commented on pull request #34628: URL: https://github.com/apache/spark/pull/34628#issuecomment-974863541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

  1   2   3   4   >