[GitHub] [spark] SparkQA commented on pull request #34364: [SPARK-37073][AVRO][TEST] Pass all UTs in `external/avro` with Java 17
SparkQA commented on pull request #34364: URL: https://github.com/apache/spark/pull/34364#issuecomment-949309560 **[Test build #144530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144530/testReport)** for PR 34364 at commit [`a0c7118`](https://github.com/apache/spark/commit/a0c7118cf3a48f0f465b1f0cd949a300e7345282). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang opened a new pull request #34364: [SPARK-37073][AVRO][TEST] Pass all UTs in `external/avro` with Java 17
LuciferYang opened a new pull request #34364: URL: https://github.com/apache/spark/pull/34364 ### What changes were proposed in this pull request? The `external/avro` module test with Java 17 has 2 UT failed, the reason for the failure is that the content of the error message used for assertion has changed from ``` Caused by: java.lang.NullPointerException: null of string in string in field Name of test_schema in test_schema ``` to ``` Caused by: java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "datum" is null of string in string in field Name of test_schema in test_schema ``` This change should be caused by `JEP 358: Helpful NullPointerExceptions`. So the change of this pr is splits one assertion into two assertions to be compatible with different Java versions. ### Why are the changes needed? Pass UT with JDK 17 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass the Jenkins or GitHub Action - Manual test use Java 17 `mvn clean install -pl external/avro` **Before** ``` Run completed in 49 seconds, 212 milliseconds. Total number of tests run: 283 Suites: completed 14, aborted 0 Tests: succeeded 281, failed 2, canceled 0, ignored 2, pending 0 *** 2 TESTS FAILED *** ``` **After** ``` Run completed in 42 seconds, 825 milliseconds. Total number of tests run: 283 Suites: completed 14, aborted 0 Tests: succeeded 283, failed 0, canceled 0, ignored 2, pending 0 All tests passed. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
AmplabJenkins removed a comment on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-949304531 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144527/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
SparkQA commented on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949305726 **[Test build #144529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144529/testReport)** for PR 34362 at commit [`affe2f6`](https://github.com/apache/spark/commit/affe2f64457c67c133e45cfd6de372649d10e2cb). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34238: [SPARK-36969][PYTHON] Inline type hints for SparkContext
AmplabJenkins removed a comment on pull request #34238: URL: https://github.com/apache/spark/pull/34238#issuecomment-949304530 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144528/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
AmplabJenkins removed a comment on pull request #34337: URL: https://github.com/apache/spark/pull/34337#issuecomment-949304532 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48997/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
AmplabJenkins commented on pull request #34337: URL: https://github.com/apache/spark/pull/34337#issuecomment-949304532 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48997/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34238: [SPARK-36969][PYTHON] Inline type hints for SparkContext
AmplabJenkins commented on pull request #34238: URL: https://github.com/apache/spark/pull/34238#issuecomment-949304530 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144528/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
AmplabJenkins commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-949304531 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144527/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34238: [SPARK-36969][PYTHON] Inline type hints for SparkContext
SparkQA removed a comment on pull request #34238: URL: https://github.com/apache/spark/pull/34238#issuecomment-949284374 **[Test build #144528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144528/testReport)** for PR 34238 at commit [`45e0406`](https://github.com/apache/spark/commit/45e0406c842f9787c2a376a2431677865301817c). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
SparkQA removed a comment on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-949284252 **[Test build #144527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144527/testReport)** for PR 34363 at commit [`1eadd75`](https://github.com/apache/spark/commit/1eadd750b1bb8e24cc658076d858ca2a718cade2). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34292: [SPARK-37017][SQL] Reduce the scope of synchronized to prevent potential deadlock
HyukjinKwon commented on pull request #34292: URL: https://github.com/apache/spark/pull/34292#issuecomment-949302897 Otherwise, I am going to propose revert this because this patch apparently has not been tested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34238: [SPARK-36969][PYTHON] Inline type hints for SparkContext
SparkQA commented on pull request #34238: URL: https://github.com/apache/spark/pull/34238#issuecomment-949302633 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48999/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17
LuciferYang commented on pull request #34313: URL: https://github.com/apache/spark/pull/34313#issuecomment-949302214 thanks all ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
SparkQA commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-949300193 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48998/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
wangyum commented on a change in pull request #34362:
URL: https://github.com/apache/spark/pull/34362#discussion_r734242476
##
File path: dev/deps/spark-deps-hadoop-3.2-hive-2.3
##
@@ -120,6 +120,7 @@ jakarta.ws.rs-api/2.1.6//jakarta.ws.rs-api-2.1.6.jar
jakarta.xml.bind-api/2.3.2//jakarta.xml.bind-api-2.3.2.jar
janino/3.0.16//janino-3.0.16.jar
javassist/3.25.0-GA//javassist-3.25.0-GA.jar
+javax.annotation-api/1.3.2//javax.annotation-api-1.3.2.jar
Review comment:
I have removed javax.annotation-api-1.3.2.jar. It seems we do not need
it:
```
LM-SHC-16508156:thrift yumwang$ grep -ER "javax.annotation" .
./lib/java/gradle/environment.gradle:ext.javaxAnnotationVersion =
property('javax.annotation.version')
./lib/java/gradle/environment.gradle:compile
"javax.annotation:javax.annotation-api:${javaxAnnotationVersion}"
Binary file ./lib/java/.gradle/5.6.4/executionHistory/executionHistory.bin
matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/classAnalysis.bin matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/taskHistory.bin matches
./lib/java/gradle.properties:javax.annotation.version=1.3.2
./lib/java/build/poms/pom-default.xml:
javax.annotation
./lib/java/build/poms/pom-default.xml:
javax.annotation-api
Binary file ./lib/java/build/deps/javax.annotation-api-1.3.2.jar matches
./lib/java/build/tmp/javadoc/javadoc.options:-classpath
'/Users/yumwang/opensource/thrift/lib/java/build/classes/java/main:/Users/yumwang/opensource/thrift/lib/java/build/resources/main:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.28/2cd9b264f76e3d087ee21bfc99305928e1bdb443/slf4j-api-1.7.28.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5.10/7ca2e4276f4ef95e4db725a8cd4a1d1e7585b9e5/httpclient-4.5.10.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.servlet/javax.servlet-api/4.0.1/a27082684a2ff0bf397666c3943496c44541d1ca/javax.servlet-api-4.0.1.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar:/Users/yumwang/.gradle/caches/modules
-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.11/3acb4705652e16236558f0f4f2192cc33c3bd189/commons-codec-1.11.jar'
./compiler/cpp/src/thrift/generate/t_java_generator.cc: indent(out) <<
"@javax.annotation.Generated(value = \"" << autogen_summary() << "\"";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
wangyum commented on a change in pull request #34362:
URL: https://github.com/apache/spark/pull/34362#discussion_r734242476
##
File path: dev/deps/spark-deps-hadoop-3.2-hive-2.3
##
@@ -120,6 +120,7 @@ jakarta.ws.rs-api/2.1.6//jakarta.ws.rs-api-2.1.6.jar
jakarta.xml.bind-api/2.3.2//jakarta.xml.bind-api-2.3.2.jar
janino/3.0.16//janino-3.0.16.jar
javassist/3.25.0-GA//javassist-3.25.0-GA.jar
+javax.annotation-api/1.3.2//javax.annotation-api-1.3.2.jar
Review comment:
I removed javax.annotation-api-1.3.2.jar. It seems we do not need it:
```
LM-SHC-16508156:thrift yumwang$ grep -ER "javax.annotation" .
./lib/java/gradle/environment.gradle:ext.javaxAnnotationVersion =
property('javax.annotation.version')
./lib/java/gradle/environment.gradle:compile
"javax.annotation:javax.annotation-api:${javaxAnnotationVersion}"
Binary file ./lib/java/.gradle/5.6.4/executionHistory/executionHistory.bin
matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/classAnalysis.bin matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/taskHistory.bin matches
./lib/java/gradle.properties:javax.annotation.version=1.3.2
./lib/java/build/poms/pom-default.xml:
javax.annotation
./lib/java/build/poms/pom-default.xml:
javax.annotation-api
Binary file ./lib/java/build/deps/javax.annotation-api-1.3.2.jar matches
./lib/java/build/tmp/javadoc/javadoc.options:-classpath
'/Users/yumwang/opensource/thrift/lib/java/build/classes/java/main:/Users/yumwang/opensource/thrift/lib/java/build/resources/main:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.28/2cd9b264f76e3d087ee21bfc99305928e1bdb443/slf4j-api-1.7.28.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5.10/7ca2e4276f4ef95e4db725a8cd4a1d1e7585b9e5/httpclient-4.5.10.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.servlet/javax.servlet-api/4.0.1/a27082684a2ff0bf397666c3943496c44541d1ca/javax.servlet-api-4.0.1.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar:/Users/yumwang/.gradle/caches/modules
-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.11/3acb4705652e16236558f0f4f2192cc33c3bd189/commons-codec-1.11.jar'
./compiler/cpp/src/thrift/generate/t_java_generator.cc: indent(out) <<
"@javax.annotation.Generated(value = \"" << autogen_summary() << "\"";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
wangyum commented on a change in pull request #34362:
URL: https://github.com/apache/spark/pull/34362#discussion_r734238451
##
File path: pom.xml
##
@@ -2442,6 +2442,10 @@
org.slf4j
slf4j-api
+
+javax.annotation
+javax.annotation-api
+
Review comment:
It seems we do not need it:
```
LM-SHC-16508156:thrift yumwang$ grep -ER "javax.annotation" .
./lib/java/gradle/environment.gradle:ext.javaxAnnotationVersion =
property('javax.annotation.version')
./lib/java/gradle/environment.gradle:compile
"javax.annotation:javax.annotation-api:${javaxAnnotationVersion}"
Binary file ./lib/java/.gradle/5.6.4/executionHistory/executionHistory.bin
matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/classAnalysis.bin matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/taskHistory.bin matches
./lib/java/gradle.properties:javax.annotation.version=1.3.2
./lib/java/build/poms/pom-default.xml:
javax.annotation
./lib/java/build/poms/pom-default.xml:
javax.annotation-api
Binary file ./lib/java/build/deps/javax.annotation-api-1.3.2.jar matches
./lib/java/build/tmp/javadoc/javadoc.options:-classpath
'/Users/yumwang/opensource/thrift/lib/java/build/classes/java/main:/Users/yumwang/opensource/thrift/lib/java/build/resources/main:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.28/2cd9b264f76e3d087ee21bfc99305928e1bdb443/slf4j-api-1.7.28.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5.10/7ca2e4276f4ef95e4db725a8cd4a1d1e7585b9e5/httpclient-4.5.10.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.servlet/javax.servlet-api/4.0.1/a27082684a2ff0bf397666c3943496c44541d1ca/javax.servlet-api-4.0.1.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar:/Users/yumwang/.gradle/caches/modules
-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.11/3acb4705652e16236558f0f4f2192cc33c3bd189/commons-codec-1.11.jar'
./compiler/cpp/src/thrift/generate/t_java_generator.cc: indent(out) <<
"@javax.annotation.Generated(value = \"" << autogen_summary() << "\"";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
HyukjinKwon commented on a change in pull request #34363: URL: https://github.com/apache/spark/pull/34363#discussion_r734240778 ## File path: python/pyspark/accumulators.py ## @@ -176,44 +193,44 @@ class AccumulatorParam(object): [7.0, 8.0, 9.0] """ -def zero(self, value): +def zero(self, value: T) -> T: """ Provide a "zero value" for the type, compatible in dimensions with the provided `value` (e.g., a zero vector) """ raise NotImplementedError -def addInPlace(self, value1, value2): +def addInPlace(self, value1: T, value2: T) -> T: """ Add two values of the accumulator's data type, returning a new value; for efficiency, can also update `value1` in place and return it. """ raise NotImplementedError -class AddingAccumulatorParam(AccumulatorParam): +class AddingAccumulatorParam(AccumulatorParam[U]): """ An AccumulatorParam that uses the + operators to add values. Designed for simple types such as integers, floats, and lists. Requires the zero value for the underlying type as a parameter. """ -def __init__(self, zero_value): +def __init__(self, zero_value: U): self.zero_value = zero_value -def zero(self, value): +def zero(self, value: U) -> U: return self.zero_value -def addInPlace(self, value1, value2): -value1 += value2 +def addInPlace(self, value1: U, value2: U) -> U: +value1 += value2 # type: ignore[operator] Review comment: @dchvn let's make sure explain why these are ignored to make the review easier. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
HyukjinKwon commented on a change in pull request #34363: URL: https://github.com/apache/spark/pull/34363#discussion_r734240526 ## File path: python/pyspark/accumulators.py ## @@ -20,20 +20,32 @@ import struct import socketserver as SocketServer import threading +from typing import TypeVar, Generic, Tuple, Callable, Type, Dict, Union, TYPE_CHECKING + from pyspark.serializers import read_int, PickleSerializer +if TYPE_CHECKING: +from pyspark._typing import SupportsIAdd +import socketserver.BaseRequestHandler # type: ignore Review comment: why do we need ignore here? ## File path: python/pyspark/accumulators.py ## @@ -264,7 +281,12 @@ def authenticate_and_accum_updates(): class AccumulatorServer(SocketServer.TCPServer): -def __init__(self, server_address, RequestHandlerClass, auth_token): +def __init__( +self, +server_address: Tuple[str, int], +RequestHandlerClass: Type["socketserver.BaseRequestHandler"], Review comment: what's diff between `Type["socketserver.BaseRequestHandler"]` and `"socketserver.BaseRequestHandler"`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34238: [SPARK-36969][PYTHON] Inline type hints for SparkContext
SparkQA commented on pull request #34238: URL: https://github.com/apache/spark/pull/34238#issuecomment-949294399 **[Test build #144528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144528/testReport)** for PR 34238 at commit [`45e0406`](https://github.com/apache/spark/commit/45e0406c842f9787c2a376a2431677865301817c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
wangyum commented on a change in pull request #34362:
URL: https://github.com/apache/spark/pull/34362#discussion_r734240298
##
File path:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java
##
@@ -45,11 +45,12 @@ public TSetIpAddressProcessor(Iface iface) {
}
@Override
- public boolean process(final TProtocol in, final TProtocol out) throws
TException {
+ public void process(final TProtocol in, final TProtocol out) throws
TException {
Review comment:
This change copied from Hive:
https://github.com/apache/hive/commit/1945e2f67e5b09cdda40146b87e1ba492f897196#diff-8f288ca198b08c9f716e68a6743ae8937b4740b69f9019c10925fb8d8083a0b0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
SparkQA commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-949294194 **[Test build #144527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144527/testReport)** for PR 34363 at commit [`1eadd75`](https://github.com/apache/spark/commit/1eadd750b1bb8e24cc658076d858ca2a718cade2). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Accumulator(Generic[T]):` * `class AccumulatorParam(Generic[T]):` * `class AddingAccumulatorParam(AccumulatorParam[U]):` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
HyukjinKwon commented on a change in pull request #34323:
URL: https://github.com/apache/spark/pull/34323#discussion_r734239196
##
File path: python/pyspark/streaming/kinesis.py
##
@@ -14,17 +14,21 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
+from typing import TypeVar, Optional, Callable
from pyspark.serializers import NoOpSerializer
from pyspark.storagelevel import StorageLevel
from pyspark.streaming import DStream
-from pyspark.util import _print_missing_jar
+from pyspark.streaming.context import StreamingContext
+from pyspark.util import _print_missing_jar # type: ignore[attr-defined]
__all__ = ['KinesisUtils', 'InitialPositionInStream', 'utf8_decoder']
+T = TypeVar("T")
Review comment:
Can you see if it works if you set `covariant=True`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
wangyum commented on a change in pull request #34362:
URL: https://github.com/apache/spark/pull/34362#discussion_r734238451
##
File path: pom.xml
##
@@ -2442,6 +2442,10 @@
org.slf4j
slf4j-api
+
+javax.annotation
+javax.annotation-api
+
Review comment:
It seems we do not need it:
```
LM-SHC-16508156:thrift yumwang$ grep -ER "javax.annotation" .
./lib/java/gradle/environment.gradle:ext.javaxAnnotationVersion =
property('javax.annotation.version')
./lib/java/gradle/environment.gradle:compile
"javax.annotation:javax.annotation-api:${javaxAnnotationVersion}"
Binary file ./lib/java/.gradle/5.6.4/executionHistory/executionHistory.bin
matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/classAnalysis.bin matches
Binary file ./lib/java/.gradle/5.6.4/javaCompile/taskHistory.bin matches
./lib/java/gradle.properties:javax.annotation.version=1.3.2
./lib/java/build/poms/pom-default.xml:
javax.annotation
./lib/java/build/poms/pom-default.xml:
javax.annotation-api
Binary file ./lib/java/build/deps/javax.annotation-api-1.3.2.jar matches
./lib/java/build/tmp/javadoc/javadoc.options:-classpath
'/Users/yumwang/opensource/thrift/lib/java/build/classes/java/main:/Users/yumwang/opensource/thrift/lib/java/build/resources/main:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.28/2cd9b264f76e3d087ee21bfc99305928e1bdb443/slf4j-api-1.7.28.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5.10/7ca2e4276f4ef95e4db725a8cd4a1d1e7585b9e5/httpclient-4.5.10.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.servlet/javax.servlet-api/4.0.1/a27082684a2ff0bf397666c3943496c44541d1ca/javax.servlet-api-4.0.1.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar:/Users/yumwang/.gradle/caches/modules
-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar:/Users/yumwang/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.11/3acb4705652e16236558f0f4f2192cc33c3bd189/commons-codec-1.11.jar'
./compiler/cpp/src/thrift/generate/t_java_generator.cc: indent(out) <<
"@javax.annotation.Generated(value = \"" << autogen_summary() << "\"";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
SparkQA commented on pull request #34337: URL: https://github.com/apache/spark/pull/34337#issuecomment-949291586 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48997/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
AmplabJenkins removed a comment on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949285303 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144525/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
AmplabJenkins commented on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949285303 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144525/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen closed pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17
srowen closed pull request #34313: URL: https://github.com/apache/spark/pull/34313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17
srowen commented on pull request #34313: URL: https://github.com/apache/spark/pull/34313#issuecomment-949284674 I copied the release notes change to the JIRA too. I'll merge to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
SparkQA removed a comment on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949243242 **[Test build #144525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144525/testReport)** for PR 34241 at commit [`cabe392`](https://github.com/apache/spark/commit/cabe392635a9113ca58d4fee4929191f3cd19667). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
srowen commented on a change in pull request #34362:
URL: https://github.com/apache/spark/pull/34362#discussion_r734232033
##
File path:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java
##
@@ -45,11 +45,12 @@ public TSetIpAddressProcessor(Iface iface) {
}
@Override
- public boolean process(final TProtocol in, final TProtocol out) throws
TException {
+ public void process(final TProtocol in, final TProtocol out) throws
TException {
Review comment:
I have no idea, does this cause problems across hive versions, if we
modify this copy of the Hive code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
SparkQA commented on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949284457 **[Test build #144525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144525/testReport)** for PR 34241 at commit [`cabe392`](https://github.com/apache/spark/commit/cabe392635a9113ca58d4fee4929191f3cd19667). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34238: [SPARK-36969][PYTHON] Inline type hints for SparkContext
SparkQA commented on pull request #34238: URL: https://github.com/apache/spark/pull/34238#issuecomment-949284374 **[Test build #144528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144528/testReport)** for PR 34238 at commit [`45e0406`](https://github.com/apache/spark/commit/45e0406c842f9787c2a376a2431677865301817c). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
srowen commented on a change in pull request #34362: URL: https://github.com/apache/spark/pull/34362#discussion_r734231884 ## File path: dev/deps/spark-deps-hadoop-3.2-hive-2.3 ## @@ -120,6 +120,7 @@ jakarta.ws.rs-api/2.1.6//jakarta.ws.rs-api-2.1.6.jar jakarta.xml.bind-api/2.3.2//jakarta.xml.bind-api-2.3.2.jar janino/3.0.16//janino-3.0.16.jar javassist/3.25.0-GA//javassist-3.25.0-GA.jar +javax.annotation-api/1.3.2//javax.annotation-api-1.3.2.jar Review comment: Minor but legitimate thing we should do: update LICENSE-binary to add a line about the licensing of this, which is dual licensed as CDDL 1.1 and GPL + classpath. Just add one line around... ``` Common Development and Distribution License (CDDL) 1.1 -- javax.el:javax.el-apihttps://javaee.github.io/uel-ri/ javax.servlet.jsp:jsp-api ``` I suppose we need a copy of the license text in licenses-binary too, ideally https://github.com/javaee/javax.annotation/blob/master/LICENSE -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
SparkQA commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-949284252 **[Test build #144527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144527/testReport)** for PR 34363 at commit [`1eadd75`](https://github.com/apache/spark/commit/1eadd750b1bb8e24cc658076d858ca2a718cade2). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
AmplabJenkins removed a comment on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949283731 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144522/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34346: [SPARK-36645][SQL][FOLLOWUP] Disable min/max push down for Parquet Binary
AmplabJenkins removed a comment on pull request #34346: URL: https://github.com/apache/spark/pull/34346#issuecomment-949283727 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144520/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
AmplabJenkins removed a comment on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949283729 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48996/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
AmplabJenkins commented on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949283729 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48996/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
AmplabJenkins commented on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949283731 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144522/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34346: [SPARK-36645][SQL][FOLLOWUP] Disable min/max push down for Parquet Binary
AmplabJenkins commented on pull request #34346: URL: https://github.com/apache/spark/pull/34346#issuecomment-949283727 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144520/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dchvn commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
dchvn commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-949283715 CC @xinrong-databricks @HyukjinKwon @ueshin @zero323. FYI. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34346: [SPARK-36645][SQL][FOLLOWUP] Disable min/max push down for Parquet Binary
SparkQA removed a comment on pull request #34346: URL: https://github.com/apache/spark/pull/34346#issuecomment-949083066 **[Test build #144520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144520/testReport)** for PR 34346 at commit [`93225b0`](https://github.com/apache/spark/commit/93225b04b3736269e210fc4a3867955cc2e5699b). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
SparkQA removed a comment on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949192271 **[Test build #144522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144522/testReport)** for PR 34362 at commit [`8d9e375`](https://github.com/apache/spark/commit/8d9e3757055426241c9c17b2199608092d9e42f7). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dchvn commented on a change in pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
dchvn commented on a change in pull request #34323: URL: https://github.com/apache/spark/pull/34323#discussion_r734229556 ## File path: python/pyspark/streaming/kinesis.py ## @@ -34,11 +38,22 @@ def utf8_decoder(s): class KinesisUtils(object): @staticmethod -def createStream(ssc, kinesisAppName, streamName, endpointUrl, regionName, - initialPositionInStream, checkpointInterval, - storageLevel=StorageLevel.MEMORY_AND_DISK_2, - awsAccessKeyId=None, awsSecretKey=None, decoder=utf8_decoder, - stsAssumeRoleArn=None, stsSessionName=None, stsExternalId=None): +def createStream( +ssc: StreamingContext, +kinesisAppName: str, +streamName: str, +endpointUrl: str, +regionName: str, +initialPositionInStream: str, +checkpointInterval: int, +storageLevel: StorageLevel = StorageLevel.MEMORY_AND_DISK_2, +awsAccessKeyId: Optional[str] = None, +awsSecretKey: Optional[str] = None, +decoder: Callable[[Optional[bytes]], T] = utf8_decoder, # type: ignore[assignment] Review comment: Oh, because we have ```utf8_decoder: Callable[[Optional[bytes]], Optional[str]]``` but ```decoder``` is ```Callable[[Optional[bytes]], T]```. Do you have any suggestion? I am facing with many errors like that in ```python/pyspark/rdd.py```. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
HyukjinKwon commented on pull request #34337: URL: https://github.com/apache/spark/pull/34337#issuecomment-949280699 @AngersZh can you fill the PR description on "How was this patch tested?"? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #34346: [SPARK-36645][SQL][FOLLOWUP] Disable min/max push down for Parquet Binary
huaxingao commented on pull request #34346: URL: https://github.com/apache/spark/pull/34346#issuecomment-949280509 Thank you so much, everyone! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34357: [SPARK-37086][R][ML][TESTS] Fix the R test of FPGrowthModel for Scala 2.13
HyukjinKwon closed pull request #34357: URL: https://github.com/apache/spark/pull/34357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34357: [SPARK-37086][R][ML][TESTS] Fix the R test of FPGrowthModel for Scala 2.13
HyukjinKwon commented on pull request #34357: URL: https://github.com/apache/spark/pull/34357#issuecomment-949280270 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34315: [SPARK-37050][PYTHON] Update Conda installation instructions
HyukjinKwon closed pull request #34315: URL: https://github.com/apache/spark/pull/34315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34315: [SPARK-37050][PYTHON] Update Conda installation instructions
HyukjinKwon commented on pull request #34315: URL: https://github.com/apache/spark/pull/34315#issuecomment-949279875 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34346: [SPARK-36645][SQL][FOLLOWUP] Disable min/max push down for Parquet Binary
SparkQA commented on pull request #34346: URL: https://github.com/apache/spark/pull/34346#issuecomment-949279206 **[Test build #144520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144520/testReport)** for PR 34346 at commit [`93225b0`](https://github.com/apache/spark/commit/93225b04b3736269e210fc4a3867955cc2e5699b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
HyukjinKwon closed pull request #34360: URL: https://github.com/apache/spark/pull/34360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
HyukjinKwon commented on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-94927 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
SparkQA commented on pull request #34337: URL: https://github.com/apache/spark/pull/34337#issuecomment-949278901 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48997/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
HyukjinKwon commented on a change in pull request #34323: URL: https://github.com/apache/spark/pull/34323#discussion_r734226805 ## File path: python/pyspark/streaming/kinesis.py ## @@ -34,11 +38,22 @@ def utf8_decoder(s): class KinesisUtils(object): @staticmethod -def createStream(ssc, kinesisAppName, streamName, endpointUrl, regionName, - initialPositionInStream, checkpointInterval, - storageLevel=StorageLevel.MEMORY_AND_DISK_2, - awsAccessKeyId=None, awsSecretKey=None, decoder=utf8_decoder, - stsAssumeRoleArn=None, stsSessionName=None, stsExternalId=None): +def createStream( +ssc: StreamingContext, +kinesisAppName: str, +streamName: str, +endpointUrl: str, +regionName: str, +initialPositionInStream: str, +checkpointInterval: int, +storageLevel: StorageLevel = StorageLevel.MEMORY_AND_DISK_2, +awsAccessKeyId: Optional[str] = None, +awsSecretKey: Optional[str] = None, +decoder: Callable[[Optional[bytes]], T] = utf8_decoder, # type: ignore[assignment] Review comment: why does it need `ignore[assignment]`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34346: [SPARK-36645][SQL][FOLLOWUP] Disable min/max push down for Parquet Binary
HyukjinKwon closed pull request #34346: URL: https://github.com/apache/spark/pull/34346 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34346: [SPARK-36645][SQL][FOLLOWUP] Disable min/max push down for Parquet Binary
HyukjinKwon commented on pull request #34346: URL: https://github.com/apache/spark/pull/34346#issuecomment-949277835 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dchvn opened a new pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py
dchvn opened a new pull request #34363: URL: https://github.com/apache/spark/pull/34363 ### What changes were proposed in this pull request? Inline type hints for python/pyspark/accumulators.py ### Why are the changes needed? We can take advantage of static type checking within the functions by inlining the type hints. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
SparkQA commented on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949275575 **[Test build #144522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144522/testReport)** for PR 34362 at commit [`8d9e375`](https://github.com/apache/spark/commit/8d9e3757055426241c9c17b2199608092d9e42f7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
SparkQA commented on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949273919 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48996/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #34350: [SPARK-37081][SQL][TESTS] Upgrade the version of RDBMS and corresponding JDBC drivers used by docker-integration-tests
sarutak commented on pull request #34350: URL: https://github.com/apache/spark/pull/34350#issuecomment-949269846 cc: @gaborgsomogyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
AmplabJenkins removed a comment on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949263886 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144523/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
AmplabJenkins commented on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949263886 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144523/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
SparkQA removed a comment on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949192281 **[Test build #144523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144523/testReport)** for PR 34360 at commit [`b5c7054`](https://github.com/apache/spark/commit/b5c7054cacb0167033bc45ffd0946c28cde6eabe). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
SparkQA commented on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949263364 **[Test build #144523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144523/testReport)** for PR 34360 at commit [`b5c7054`](https://github.com/apache/spark/commit/b5c7054cacb0167033bc45ffd0946c28cde6eabe). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
SparkQA commented on pull request #34337: URL: https://github.com/apache/spark/pull/34337#issuecomment-949262509 **[Test build #144526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144526/testReport)** for PR 34337 at commit [`938a41e`](https://github.com/apache/spark/commit/938a41ed8ed5b758930e646f4bb433573c91a98b). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
AmplabJenkins removed a comment on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949262279 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48993/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
AmplabJenkins commented on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949262279 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48993/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
AmplabJenkins commented on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949262280 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48995/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
AmplabJenkins removed a comment on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949262280 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48995/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
AmplabJenkins removed a comment on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949262281 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48994/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
AmplabJenkins commented on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949262281 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48994/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
SparkQA commented on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949259574 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48994/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
SparkQA commented on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949258708 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48996/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dchvn commented on a change in pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
dchvn commented on a change in pull request #34323: URL: https://github.com/apache/spark/pull/34323#discussion_r734210281 ## File path: python/pyspark/streaming/kinesis.py ## @@ -34,11 +38,22 @@ def utf8_decoder(s): class KinesisUtils(object): @staticmethod -def createStream(ssc, kinesisAppName, streamName, endpointUrl, regionName, - initialPositionInStream, checkpointInterval, - storageLevel=StorageLevel.MEMORY_AND_DISK_2, - awsAccessKeyId=None, awsSecretKey=None, decoder=utf8_decoder, - stsAssumeRoleArn=None, stsSessionName=None, stsExternalId=None): +def createStreamcreateStream( Review comment: Updated, thank you ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
SparkQA commented on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949255321 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48995/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
SparkQA commented on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949252676 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48993/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
AngersZh commented on a change in pull request #34337:
URL: https://github.com/apache/spark/pull/34337#discussion_r734206219
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
##
@@ -586,10 +586,11 @@ object QueryExecutionErrors {
new QueryExecutionException(message, e)
}
- def cannotReadParquetFilesError(e: Exception): Throwable = {
-val message = "Encounter error while reading parquet files. " +
- "One possible cause: Parquet column cannot be converted in the " +
- "corresponding files. Details: "
+ def cannotReadParquetFilesError(
+ e: Exception,
+ path: String,
+ notice: String): Throwable = {
Review comment:
Done
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
##
@@ -586,10 +586,11 @@ object QueryExecutionErrors {
new QueryExecutionException(message, e)
}
- def cannotReadParquetFilesError(e: Exception): Throwable = {
-val message = "Encounter error while reading parquet files. " +
- "One possible cause: Parquet column cannot be converted in the " +
- "corresponding files. Details: "
+ def cannotReadParquetFilesError(
+ e: Exception,
+ path: String,
+ notice: String): Throwable = {
+val message = s"Encounter error while reading parquet file $path. $notice
Details: "
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34337: [SPARK-37066][SQL] Improve error message to show file path when failed to read next file
AngersZh commented on pull request #34337: URL: https://github.com/apache/spark/pull/34337#issuecomment-949251906 > Thank you for generalizing and updating, @AngersZh . Could you check the UT failures? It looks like wrapping exceptions requires us revise the test case asserts. > > ``` > [info] - Logical type: Decimal with too large precision *** FAILED *** (65 milliseconds) > [info] "Encounter error while reading parquet file file:///home/runner/work/spark/spark/target/tmp/spark-234e31e3-7e34-47a5-90bf-5f716c0e1e6b.avro. Details: " did not contain "Unscaled value too large for precision" (AvroLogicalTypeSuite.scala:438) > ``` Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
SparkQA commented on pull request #34241: URL: https://github.com/apache/spark/pull/34241#issuecomment-949243242 **[Test build #144525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144525/testReport)** for PR 34241 at commit [`cabe392`](https://github.com/apache/spark/commit/cabe392635a9113ca58d4fee4929191f3cd19667). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down
AmplabJenkins removed a comment on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-949242840 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144519/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down
AmplabJenkins commented on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-949242840 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144519/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34356: [SPARK-36554][PYTHON] Expose make_date expression in functions.scala
AmplabJenkins removed a comment on pull request #34356: URL: https://github.com/apache/spark/pull/34356#issuecomment-949142458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
AmplabJenkins removed a comment on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949242139 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144524/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down
SparkQA commented on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-949242065 **[Test build #144519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144519/testReport)** for PR 34291 at commit [`836746e`](https://github.com/apache/spark/commit/836746e59128491000a6e42a274aa1fc3e9ac2ca). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34356: [SPARK-36554][PYTHON] Expose make_date expression in functions.scala
AmplabJenkins commented on pull request #34356: URL: https://github.com/apache/spark/pull/34356#issuecomment-949242138 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48992/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34320: [SPARK-18621][PYTHON] make sql type reprs eval-able
AmplabJenkins commented on pull request #34320: URL: https://github.com/apache/spark/pull/34320#issuecomment-949242152 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down
SparkQA removed a comment on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-949028566 **[Test build #144519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144519/testReport)** for PR 34291 at commit [`836746e`](https://github.com/apache/spark/commit/836746e59128491000a6e42a274aa1fc3e9ac2ca). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
AmplabJenkins commented on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949242139 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144524/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
SparkQA removed a comment on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949192690 **[Test build #144524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144524/testReport)** for PR 34323 at commit [`93da9be`](https://github.com/apache/spark/commit/93da9be8df0dea11c699033a740cd13fc7722455). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
SparkQA commented on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949239343 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48995/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34356: [SPARK-36554][PYTHON] Expose make_date expression in functions.scala
SparkQA commented on pull request #34356: URL: https://github.com/apache/spark/pull/34356#issuecomment-949239228 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48992/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
SparkQA commented on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949238914 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48994/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34362: [SPARK-37090][BUILD] Upgrade libthrift to avoid security vulnerabilities
SparkQA commented on pull request #34362: URL: https://github.com/apache/spark/pull/34362#issuecomment-949237162 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48993/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34355: [SPARK-37070][TEST] Pass all UTs in `mllib-local` and `mllib` with Java 17
LuciferYang commented on pull request #34355: URL: https://github.com/apache/spark/pull/34355#issuecomment-949235992 thanks @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
SparkQA commented on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949221254 **[Test build #144524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144524/testReport)** for PR 34323 at commit [`93da9be`](https://github.com/apache/spark/commit/93da9be8df0dea11c699033a740cd13fc7722455). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34360: [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available
SparkQA commented on pull request #34360: URL: https://github.com/apache/spark/pull/34360#issuecomment-949192281 **[Test build #144523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144523/testReport)** for PR 34360 at commit [`b5c7054`](https://github.com/apache/spark/commit/b5c7054cacb0167033bc45ffd0946c28cde6eabe). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34323: [SPARK-37042][PYTHON] Inline type hints for kinesis.py and listener.py in python/pyspark/streaming
SparkQA commented on pull request #34323: URL: https://github.com/apache/spark/pull/34323#issuecomment-949192690 **[Test build #144524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144524/testReport)** for PR 34323 at commit [`93da9be`](https://github.com/apache/spark/commit/93da9be8df0dea11c699033a740cd13fc7722455). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
