[GitHub] SparkQA commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
SparkQA commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455760048 **[Test build #101429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101429/testReport)** for PR 23260 at commit [`0353a04`](https://github.com/apache/spark/commit/0353a04b1d6bcdee26fa4a2369cd226afdef1f15). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] 10110346 commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes
10110346 commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes URL: https://github.com/apache/spark/pull/23583#issuecomment-455759904 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HeartSaVioR commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
HeartSaVioR commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455759736 retest this, please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
dongjoon-hyun commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r249238262
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
##
@@ -83,10 +84,14 @@ trait CreateHiveTableAsSelectBase extends
DataWritingCommand {
tableDesc: CatalogTable,
tableExists: Boolean): DataWritingCommand
+ // A subclass should override this with the Class of the concrete type
expected to be
+ // returned from `getWritingCommand`.
+ def writingCommandClass: Class[_]
Review comment:
@rednaxelafx . Do we need to return the class itself? Can we simply return
`String` instead of `Class[_]` because we require only `name` and don't use the
class actually.
```scala
- def writingCommandClass: Class[_]
+ def writingCommandClassName: String
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] inpefess commented on a change in pull request #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1
inpefess commented on a change in pull request #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1 URL: https://github.com/apache/spark/pull/20691#discussion_r249238272 ## File path: python/pyspark/broadcast.py ## @@ -110,7 +110,7 @@ def __init__(self, sc=None, value=None, pickle_registry=None, path=None, def dump(self, value, f): try: -pickle.dump(value, f, 2) +pickle.dump(value, f, pickle.HIGHEST_PROTOCOL) Review comment: It happened here: https://github.com/apache/spark/commit/6cf507685efd01df77d663145ae08e48c7f92948#diff-bb67501acde415576c589b478e16c60aR82 Since then it never changed. I agree that there was no particular reason for that since pickle.HIGHEST_PROTOCOL in Python 2 versions is 2 for ages, not 3 or 4. Using pickle.HIGHEST_PROTOCOL consistently should be safe for that reason. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
dongjoon-hyun commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r249238262
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
##
@@ -83,10 +84,14 @@ trait CreateHiveTableAsSelectBase extends
DataWritingCommand {
tableDesc: CatalogTable,
tableExists: Boolean): DataWritingCommand
+ // A subclass should override this with the Class of the concrete type
expected to be
+ // returned from `getWritingCommand`.
+ def writingCommandClass: Class[_]
Review comment:
@rednaxelafx . Do we need to return the class itself? Can we simply return
`String` instead of `Class[_]` because we require `name` and don't use the
class actually.
```scala
- def writingCommandClass: Class[_]
+ def writingCommandClassName: String
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23583: [MINOR][TEST]Correct some unit test mistakes
AmplabJenkins removed a comment on issue #23583: [MINOR][TEST]Correct some unit test mistakes URL: https://github.com/apache/spark/pull/23583#issuecomment-455759176 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101425/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
AmplabJenkins removed a comment on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455759201 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
AmplabJenkins removed a comment on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455759202 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101424/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
SparkQA removed a comment on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455749858 **[Test build #101424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101424/testReport)** for PR 23260 at commit [`0353a04`](https://github.com/apache/spark/commit/0353a04b1d6bcdee26fa4a2369cd226afdef1f15). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23583: [MINOR][TEST]Correct some unit test mistakes
SparkQA removed a comment on issue #23583: [MINOR][TEST]Correct some unit test mistakes URL: https://github.com/apache/spark/pull/23583#issuecomment-455756065 **[Test build #101425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101425/testReport)** for PR 23583 at commit [`ffb0ca1`](https://github.com/apache/spark/commit/ffb0ca1743308f21736fe04769334c4791004fc4). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes
AmplabJenkins commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes URL: https://github.com/apache/spark/pull/23583#issuecomment-455759175 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever
AmplabJenkins removed a comment on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever URL: https://github.com/apache/spark/pull/23590#issuecomment-455759066 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
AmplabJenkins commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455759201 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever
AmplabJenkins removed a comment on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever URL: https://github.com/apache/spark/pull/23590#issuecomment-455759067 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7227/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes
AmplabJenkins commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes URL: https://github.com/apache/spark/pull/23583#issuecomment-455759176 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101425/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
AmplabJenkins commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455759202 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101424/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23583: [MINOR][TEST]Correct some unit test mistakes
AmplabJenkins removed a comment on issue #23583: [MINOR][TEST]Correct some unit test mistakes URL: https://github.com/apache/spark/pull/23583#issuecomment-455759175 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes
SparkQA commented on issue #23583: [MINOR][TEST]Correct some unit test mistakes URL: https://github.com/apache/spark/pull/23583#issuecomment-455759142 **[Test build #101425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101425/testReport)** for PR 23583 at commit [`ffb0ca1`](https://github.com/apache/spark/commit/ffb0ca1743308f21736fe04769334c4791004fc4). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS
SparkQA commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-455759143 **[Test build #101424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101424/testReport)** for PR 23260 at commit [`0353a04`](https://github.com/apache/spark/commit/0353a04b1d6bcdee26fa4a2369cd226afdef1f15). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever
SparkQA commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever URL: https://github.com/apache/spark/pull/23590#issuecomment-455759100 **[Test build #101427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101427/testReport)** for PR 23590 at commit [`da1aaac`](https://github.com/apache/spark/commit/da1aaac7cb1b0687ec2e19c5049b96e216f9dce2). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever
AmplabJenkins commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever URL: https://github.com/apache/spark/pull/23590#issuecomment-455759066 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever
AmplabJenkins commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever URL: https://github.com/apache/spark/pull/23590#issuecomment-455759067 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7227/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] inpefess commented on a change in pull request #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1
inpefess commented on a change in pull request #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1 URL: https://github.com/apache/spark/pull/20691#discussion_r249238114 ## File path: python/pyspark/serializers.py ## @@ -606,7 +604,7 @@ class PickleSerializer(FramedSerializer): """ def dumps(self, obj): -return pickle.dumps(obj, protocol) +return pickle.dumps(obj, pickle.HIGHEST_PROTOCOL) Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever
dongjoon-hyun commented on issue #23590: [SPARK-26665][Core]Fix a bug that BlockTransferService.fetchBlockSync may hang forever URL: https://github.com/apache/spark/pull/23590#issuecomment-455758843 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun closed pull request #23412: [SPARK-26477][CORE] Use ConfigEntry for hardcoded configs for unsafe category
dongjoon-hyun closed pull request #23412: [SPARK-26477][CORE] Use ConfigEntry for hardcoded configs for unsafe category URL: https://github.com/apache/spark/pull/23412 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
