[GitHub] [spark] SparkQA commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
SparkQA commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780374411 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39774/

[GitHub] [spark] cloud-fan commented on pull request #31553: [WIP][SPARK-34425][SQL] Clarify error message

2021-02-16 Thread GitBox
cloud-fan commented on pull request #31553: URL: https://github.com/apache/spark/pull/31553#issuecomment-780368362 As I mentioned in https://github.com/apache/spark/pull/31541#issuecomment-780319947 , people can extend the session catalog by a custom v2 catalog, but it must share the same

[GitHub] [spark] SparkQA commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
SparkQA commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780367663 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39773/

[GitHub] [spark] c21 commented on a change in pull request #31567: [SPARK-26138][SQL] Pushdown limit through InnerLike when condition is empty

2021-02-16 Thread GitBox
c21 commented on a change in pull request #31567: URL: https://github.com/apache/spark/pull/31567#discussion_r577379572 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -539,17 +539,22 @@ object LimitPushDown extends

[GitHub] [spark] c21 commented on a change in pull request #31567: [SPARK-26138][SQL] Pushdown limit through InnerLike when condition is empty

2021-02-16 Thread GitBox
c21 commented on a change in pull request #31567: URL: https://github.com/apache/spark/pull/31567#discussion_r577376022 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -539,17 +539,22 @@ object LimitPushDown extends

[GitHub] [spark] wangyum commented on a change in pull request #31567: [SPARK-26138][SQL] Pushdown limit through InnerLike when condition is empty

2021-02-16 Thread GitBox
wangyum commented on a change in pull request #31567: URL: https://github.com/apache/spark/pull/31567#discussion_r577372969 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -539,17 +539,22 @@ object LimitPushDown extends

[GitHub] [spark] MaxGekk commented on pull request #31564: [SPARK-34437][SQL][DOCS] Update Spark SQL guide about the rebasing DS options and SQL configs

2021-02-16 Thread GitBox
MaxGekk commented on pull request #31564: URL: https://github.com/apache/spark/pull/31564#issuecomment-780354284 > These options are for migration purposes (same as the legacy configs). I believe the rebase configs were placed to the `legacy` namespace mistakenly because they can be

[GitHub] [spark] SparkQA commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
SparkQA commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780350692 **[Test build #135193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135193/testReport)** for PR 29726 at commit

[GitHub] [spark] SparkQA commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
SparkQA commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780350074 **[Test build #135192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135192/testReport)** for PR 31574 at commit

[GitHub] [spark] viirya commented on a change in pull request #31573: [SPARK-34444][SQL] Pushdown scalar-subquery filter to FileSourceScan

2021-02-16 Thread GitBox
viirya commented on a change in pull request #31573: URL: https://github.com/apache/spark/pull/31573#discussion_r577366266 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala ## @@ -85,6 +85,26 @@ object SubqueryExpression {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780348223 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135188/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780348225 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39772/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780348226 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135187/

[GitHub] [spark] AmplabJenkins commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780348225 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39772/

[GitHub] [spark] AmplabJenkins commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780348223 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135188/

[GitHub] [spark] AmplabJenkins commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780348226 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135187/

[GitHub] [spark] HyukjinKwon commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
HyukjinKwon commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780335184 Looks like the test failure is related. The same test failed in GA action too (I retriggered after checking it).

[GitHub] [spark] cloud-fan commented on pull request #31564: [SPARK-34437][SQL][DOCS] Update Spark SQL guide about the rebasing DS options and SQL configs

2021-02-16 Thread GitBox
cloud-fan commented on pull request #31564: URL: https://github.com/apache/spark/pull/31564#issuecomment-780333787 These options are for migration purposes (same as the legacy configs). Do we really need to mention them in the public doc? How about the migration guide?

[GitHub] [spark] wangyum commented on a change in pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
wangyum commented on a change in pull request #29726: URL: https://github.com/apache/spark/pull/29726#discussion_r577353386 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/dynamicpruning/PartitionPruning.scala ## @@ -196,6 +198,25 @@ object

[GitHub] [spark] dongjoon-hyun commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
dongjoon-hyun commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780332574 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
SparkQA removed a comment on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780270050 **[Test build #135187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135187/testReport)** for PR 29726 at commit

[GitHub] [spark] SparkQA commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
SparkQA commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780332095 **[Test build #135187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135187/testReport)** for PR 29726 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577349814 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] SparkQA removed a comment on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
SparkQA removed a comment on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780285105 **[Test build #135188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135188/testReport)** for PR 31574 at commit

[GitHub] [spark] SparkQA commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
SparkQA commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780327914 **[Test build #135188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135188/testReport)** for PR 31574 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577349814 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] HyukjinKwon commented on pull request #31541: Revert "[SPARK-34209][SQL] Delegate table name validation to the session catalog"

2021-02-16 Thread GitBox
HyukjinKwon commented on pull request #31541: URL: https://github.com/apache/spark/pull/31541#issuecomment-780326808 @rdblue, can you confirm? it would be great if you point out which part of your design doc explains this.

[GitHub] [spark] SparkQA commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
SparkQA commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780326702 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39772/

[GitHub] [spark] cloud-fan commented on pull request #31541: Revert "[SPARK-34209][SQL] Delegate table name validation to the session catalog"

2021-02-16 Thread GitBox
cloud-fan commented on pull request #31541: URL: https://github.com/apache/spark/pull/31541#issuecomment-780326345 @RussellSpitzer I looked at your use case in https://github.com/apache/spark/pull/31541#issuecomment-776882137 , and I think it's a misuse of the session catalog, as you are

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577346729 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577345951 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577344791 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577344791 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780319879 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135185/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780319875 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39768/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780319876 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780319878 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135191/

[GitHub] [spark] c21 commented on a change in pull request #31477: [SPARK-34369][SQL] Track number of pairs processed out of Join

2021-02-16 Thread GitBox
c21 commented on a change in pull request #31477: URL: https://github.com/apache/spark/pull/31477#discussion_r577343739 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala ## @@ -48,6 +48,13 @@ trait HashJoin extends BaseJoinExec with

[GitHub] [spark] cloud-fan commented on pull request #31541: Revert "[SPARK-34209][SQL] Delegate table name validation to the session catalog"

2021-02-16 Thread GitBox
cloud-fan commented on pull request #31541: URL: https://github.com/apache/spark/pull/31541#issuecomment-780319947 Technically people can replace the session catalog with a custom v2 implementation by setting `spark.sql.catalog.spark_catalog`, but this is not the expected usage: ```

[GitHub] [spark] AmplabJenkins commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780319878 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135191/

[GitHub] [spark] AmplabJenkins commented on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780319876 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780319879 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135185/

[GitHub] [spark] AmplabJenkins commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780319875 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39768/

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577343559 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577342799 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577341368 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577341368 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] sarutak edited a comment on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
sarutak edited a comment on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780316712 @dongjoon-hyun Sorry for my late response. I don't think this CVE is a blocker. This is an automated

[GitHub] [spark] sarutak commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
sarutak commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780316712 @dongjoon-hyun Sorry for my late response. I don't think this CVE is not a blocker. This is an automated

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add a java implementation of simple writable data source in DataSourceV2Suite

2021-02-16 Thread GitBox
dongjoon-hyun commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r577340685 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,374 @@ +/* + * Licensed

[GitHub] [spark] c21 commented on a change in pull request #31567: [SPARK-26138][SQL] Pushdown limit through InnerLike when condition is empty

2021-02-16 Thread GitBox
c21 commented on a change in pull request #31567: URL: https://github.com/apache/spark/pull/31567#discussion_r577339415 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -539,17 +539,22 @@ object LimitPushDown extends

[GitHub] [spark] SparkQA removed a comment on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
SparkQA removed a comment on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780266532 **[Test build #135185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135185/testReport)** for PR 31565 at commit

[GitHub] [spark] SparkQA commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
SparkQA commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780314233 **[Test build #135185 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135185/testReport)** for PR 31565 at commit

[GitHub] [spark] SparkQA commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
SparkQA commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780313957 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39772/

[GitHub] [spark] dongjoon-hyun commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
dongjoon-hyun commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780313892 According to CVE-2020-27218 description, it looks like a non-blocker (as you marked in SPARK-34449). In general, Spark network environment is non-public. > The attacker

[GitHub] [spark] SparkQA commented on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
SparkQA commented on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780312455 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39771/

[GitHub] [spark] SparkQA removed a comment on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
SparkQA removed a comment on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780303150 **[Test build #135191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135191/testReport)** for PR 31572 at commit

[GitHub] [spark] SparkQA commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
SparkQA commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780307294 **[Test build #135191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135191/testReport)** for PR 31572 at commit

[GitHub] [spark] SparkQA commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
SparkQA commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780303150 **[Test build #135191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135191/testReport)** for PR 31572 at commit

[GitHub] [spark] SparkQA commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
SparkQA commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780302419 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39768/

[GitHub] [spark] dongjoon-hyun commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
dongjoon-hyun commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780301280 Thank you for pinging me, @sarutak . BTW, is this a blocker for on-going RC votes? This is an automated

[GitHub] [spark] dongjoon-hyun commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

2021-02-16 Thread GitBox
dongjoon-hyun commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-780299732 Thank you, @sarutak and all. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
SparkQA commented on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780299648 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39771/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780297921 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-780297923 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780297924 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39769/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780297918 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-780297927 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135184/

[GitHub] [spark] AmplabJenkins commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-780297927 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135184/

[GitHub] [spark] AmplabJenkins commented on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780297919 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-780297925 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780297922 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780297924 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39769/

[GitHub] [spark] SparkQA commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
SparkQA commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780296498 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39768/

[GitHub] [spark] SparkQA commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
SparkQA commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780295038 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39769/

[GitHub] [spark] SparkQA commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
SparkQA commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780293530 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39769/

[GitHub] [spark] SparkQA removed a comment on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
SparkQA removed a comment on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780246032 **[Test build #135183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135183/testReport)** for PR 31565 at commit

[GitHub] [spark] SparkQA commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
SparkQA commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780287584 **[Test build #135183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135183/testReport)** for PR 31565 at commit

[GitHub] [spark] SparkQA commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
SparkQA commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780287116 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39766/

[GitHub] [spark] SparkQA commented on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
SparkQA commented on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780287067 **[Test build #135190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135190/testReport)** for PR 30841 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
HeartSaVioR commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780286786 Thanks for the quick fix, @c21 ! I'll merge this to master/branch-3.1 in tomorrow morning in KST unless there's no further comment.

[GitHub] [spark] c21 commented on a change in pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
c21 commented on a change in pull request #31572: URL: https://github.com/apache/spark/pull/31572#discussion_r577313278 ## File path: docs/structured-streaming-programming-guide.md ## @@ -1337,7 +1337,7 @@ impressionsWithWatermark.join( clickTime >= impressionTime AND

[GitHub] [spark] SparkQA commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
SparkQA commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780285105 **[Test build #135188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135188/testReport)** for PR 31574 at commit

[GitHub] [spark] SparkQA commented on pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
SparkQA commented on pull request #30841: URL: https://github.com/apache/spark/pull/30841#issuecomment-780285032 **[Test build #135189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135189/testReport)** for PR 30841 at commit

[GitHub] [spark] c21 commented on pull request #31572: [SPARK-34446][SS][DOCS] Update doc for stream-stream join (full outer + left semi)

2021-02-16 Thread GitBox
c21 commented on pull request #31572: URL: https://github.com/apache/spark/pull/31572#issuecomment-780285057 @HeartSaVioR - updated the description for required table screenshot and addressed comment of rewording in semi join section.

[GitHub] [spark] SparkQA commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-16 Thread GitBox
SparkQA commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-780284820 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39767/

[GitHub] [spark] SparkQA commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
SparkQA commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780281857 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39766/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780279169 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39764/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-16 Thread GitBox
AmplabJenkins removed a comment on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-780279168 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39765/

[GitHub] [spark] AmplabJenkins commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780279169 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39764/

[GitHub] [spark] AmplabJenkins commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-16 Thread GitBox
AmplabJenkins commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-780279168 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39765/

[GitHub] [spark] viirya commented on a change in pull request #31567: [SPARK-26138][SQL] Pushdown limit through InnerLike when condition is empty

2021-02-16 Thread GitBox
viirya commented on a change in pull request #31567: URL: https://github.com/apache/spark/pull/31567#discussion_r577306759 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/LimitPushdownSuite.scala ## @@ -194,4 +194,22 @@ class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #30841: [SPARK-28191][SS] New data source - state - reader part

2021-02-16 Thread GitBox
HeartSaVioR commented on a change in pull request #30841: URL: https://github.com/apache/spark/pull/30841#discussion_r577305481 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/state/StateDataSourceV2.scala ## @@ -0,0 +1,112 @@ +/* + *

[GitHub] [spark] SparkQA commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-16 Thread GitBox
SparkQA commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-780276630 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39767/

[GitHub] [spark] wangyum edited a comment on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
wangyum edited a comment on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780266596 ```scala sql("set spark.sql.adaptive.enabled=false") sql("CREATE TABLE t1 using parquet partitioned by (b) AS SELECT id AS a, id % 1000 AS b FROM range(2L)

[GitHub] [spark] sarutak commented on pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
sarutak commented on pull request #31574: URL: https://github.com/apache/spark/pull/31574#issuecomment-780272740 cc: @HyukjinKwon and @dongjoon-hyun because you are release managers. This is an automated message from the

[GitHub] [spark] sarutak opened a new pull request #31574: [SPARK-34449][BUILD] Upgrade Jetty to fix CVE-2020-27218

2021-02-16 Thread GitBox
sarutak opened a new pull request #31574: URL: https://github.com/apache/spark/pull/31574 ### What changes were proposed in this pull request? This PR upgrades Jetty from `9.4.34` to `9.4.36`. ### Why are the changes needed? CVE-2020-27218 affects currently used Jetty

[GitHub] [spark] SparkQA commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

2021-02-16 Thread GitBox
SparkQA commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-780270050 **[Test build #135187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135187/testReport)** for PR 29726 at commit

[GitHub] [spark] SparkQA commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-16 Thread GitBox
SparkQA commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-780268326 **[Test build #135186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135186/testReport)** for PR 31569 at commit

[GitHub] [spark] SparkQA commented on pull request #31565: [SPARK-34438][SPARK SUBMIT] Check path component in isPython/isR, not full URI

2021-02-16 Thread GitBox
SparkQA commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-780267070 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39764/

  1   2   3   4   >