[GitHub] maropu commented on a change in pull request #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses
maropu commented on a change in pull request #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses URL: https://github.com/apache/spark/pull/20433#discussion_r24463 ## File path: sql/core/src/test/resources/sql-tests/inputs/interval.sql ## @@ -0,0 +1,185 @@ +-- Basic tests for intervals +--SET spark.sql.parser.ansi.enabled=true + +select + '1' second, + 2 seconds, Review comment: Regardless of whether seconds reserved or not, this pr currently regards the statement as interval. Anyway, IMO seconds and the other units should be reserved in the ANSI mode. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses
gatorsmile commented on a change in pull request #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses URL: https://github.com/apache/spark/pull/20433#discussion_r244629240 ## File path: sql/core/src/test/resources/sql-tests/inputs/interval.sql ## @@ -0,0 +1,185 @@ +-- Basic tests for intervals +--SET spark.sql.parser.ansi.enabled=true + +select + '1' second, + 2 seconds, Review comment: `seconds` needs to be a reserved word in the ANSI mode; otherwise, this could be either alias or interval unit? Is my understanding right? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #22954: [SPARK-25981][R] Enables Arrow optimization from R DataFrame to Spark DataFrame
HyukjinKwon commented on a change in pull request #22954: [SPARK-25981][R]
Enables Arrow optimization from R DataFrame to Spark DataFrame
URL: https://github.com/apache/spark/pull/22954#discussion_r244629053
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -1287,8 +1287,9 @@ object SQLConf {
val ARROW_EXECUTION_ENABLED =
buildConf("spark.sql.execution.arrow.enabled")
.doc("When true, make use of Apache Arrow for columnar data transfers.
Currently available " +
-"for use with pyspark.sql.DataFrame.toPandas, and " +
-"pyspark.sql.SparkSession.createDataFrame when its input is a Pandas
DataFrame. " +
+"for use with pyspark.sql.DataFrame.toPandas, " +
+"pyspark.sql.SparkSession.createDataFrame when its input is a Pandas
DataFrame," +
Review comment:
Oh yeah. I need to push additional commit when it's released. Will fix it
together later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
dongjoon-hyun commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712265 Sure. @gatorsmile . Could you rebase this and fix the build error, @peter-toth ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712072 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100609/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712071 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] maropu commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses
maropu commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses URL: https://github.com/apache/spark/pull/20433#issuecomment-450712099 I think #23259 might have much room for discussions, so how about taking this with the new ansi option first, and then moving into #23259? (Happy new year! I hope we have a good year!) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712081 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6534/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712081 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6534/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712080 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
SparkQA removed a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450711856 **[Test build #100609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100609/testReport)** for PR 22617 at commit [`3a52abc`](https://github.com/apache/spark/commit/3a52abc830751fede6ef937ff22a8964626eab1b). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712080 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
SparkQA commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712070 **[Test build #100609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100609/testReport)** for PR 22617 at commit [`3a52abc`](https://github.com/apache/spark/commit/3a52abc830751fede6ef937ff22a8964626eab1b). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712072 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100609/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
AmplabJenkins commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450712071 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] asfgit closed pull request #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
asfgit closed pull request #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter
does not handle ByteType
URL: https://github.com/apache/spark/pull/23400
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
index edea549748b47..922bef284c98e 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
@@ -438,6 +438,10 @@ object JdbcUtils extends Logging {
(rs: ResultSet, row: InternalRow, pos: Int) =>
row.setShort(pos, rs.getShort(pos + 1))
+case ByteType =>
+ (rs: ResultSet, row: InternalRow, pos: Int) =>
+row.update(pos, rs.getByte(pos + 1))
+
case StringType =>
(rs: ResultSet, row: InternalRow, pos: Int) =>
// TODO(davies): use getBytes for better performance, if the encoding
is UTF-8
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
index 71e83767964a0..e4641631e607d 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
@@ -56,6 +56,20 @@ class JDBCSuite extends QueryTest
Some(StringType)
}
+ val testH2DialectTinyInt = new JdbcDialect {
+override def canHandle(url: String): Boolean = url.startsWith("jdbc:h2")
+override def getCatalystType(
+sqlType: Int,
+typeName: String,
+size: Int,
+md: MetadataBuilder): Option[DataType] = {
+ sqlType match {
+case java.sql.Types.TINYINT => Some(ByteType)
+case _ => None
+ }
+}
+ }
+
before {
Utils.classForName("org.h2.Driver")
// Extra properties that will be specified for our database. We need these
to test
@@ -693,6 +707,17 @@ class JDBCSuite extends QueryTest
JdbcDialects.unregisterDialect(testH2Dialect)
}
+ test("Map TINYINT to ByteType via JdbcDialects") {
+JdbcDialects.registerDialect(testH2DialectTinyInt)
+val df = spark.read.jdbc(urlWithUserAndPass, "test.inttypes", new
Properties())
+val rows = df.collect()
+assert(rows.length === 2)
+assert(rows(0).get(2).isInstanceOf[Byte])
+assert(rows(0).getByte(2) === 3)
+assert(rows(1).isNullAt(2))
+JdbcDialects.unregisterDialect(testH2DialectTinyInt)
+ }
+
test("Default jdbc dialect registration") {
assert(JdbcDialects.get("jdbc:mysql://127.0.0.1/db") == MySQLDialect)
assert(JdbcDialects.get("jdbc:postgresql://127.0.0.1/db") ==
PostgresDialect)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
gatorsmile commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450711900 Please help review it @dongjoon-hyun @kiszk @wangyum This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
SparkQA commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450711856 **[Test build #100609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100609/testReport)** for PR 22617 at commit [`3a52abc`](https://github.com/apache/spark/commit/3a52abc830751fede6ef937ff22a8964626eab1b). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB
AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450711832 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB
AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450711833 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6533/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB
AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450711833 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6533/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB
AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450711832 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
gatorsmile commented on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-450711818 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
HyukjinKwon commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450711791 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711749 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100604/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711748 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711748 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
SparkQA removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450704867 **[Test build #100604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100604/testReport)** for PR 21790 at commit [`d1d4adf`](https://github.com/apache/spark/commit/d1d4adfe0620253f65e5c54366768c695823ae28). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711749 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100604/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #22936: [SPARK-19799] Support WITH clause (CTE) in subqueries
gatorsmile commented on a change in pull request #22936: [SPARK-19799] Support
WITH clause (CTE) in subqueries
URL: https://github.com/apache/spark/pull/22936#discussion_r244628855
##
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
##
@@ -535,7 +535,19 @@ class SQLQuerySuite extends QueryTest with
SharedSQLContext {
|q2 as (select * from testData where key = '4')
|select * from q1 union all select * from q2""".stripMargin),
Row(5, "5") :: Row(4, "4") :: Nil)
+ }
+ test("SPARK-19799: Support WITH clause in subqueries") {
Review comment:
Could you add more test cases? For example, can a with clause in a subquery
shadow a with clause in an enclosing query with the same name? Another example,
use with clauses in a subquery expression?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
SparkQA commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711710 **[Test build #100604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100604/testReport)** for PR 21790 at commit [`d1d4adf`](https://github.com/apache/spark/commit/d1d4adfe0620253f65e5c54366768c695823ae28). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB
SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450711701 **[Test build #100608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100608/testReport)** for PR 23393 at commit [`8cc02a7`](https://github.com/apache/spark/commit/8cc02a7ef345000c0d20ad5beec8b48595a36c35). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB
AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450294063 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB
dongjoon-hyun commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450711644 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100603/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100603/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711466 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711466 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
SparkQA removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450704730 **[Test build #100603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100603/testReport)** for PR 21790 at commit [`97493c9`](https://github.com/apache/spark/commit/97493c95e3068afe17b4665a773e8fadcd459ed3). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
SparkQA commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450711424 **[Test build #100603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100603/testReport)** for PR 21790 at commit [`97493c9`](https://github.com/apache/spark/commit/97493c95e3068afe17b4665a773e8fadcd459ed3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on a change in pull request #23416: [SPARK-26463][CORE] Use ConfigEntry for hardcoded configs for scheduler categories.
dongjoon-hyun commented on a change in pull request #23416: [SPARK-26463][CORE]
Use ConfigEntry for hardcoded configs for scheduler categories.
URL: https://github.com/apache/spark/pull/23416#discussion_r244628639
##
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##
@@ -212,11 +236,36 @@ package object config {
.toSequence
.createWithDefault(Nil)
+ private[spark] val MAX_TASK_MAX_DIRECT_RESULT_SIZE =
Review comment:
`MAX_TASK_MAX_DIRECT_RESULT_SIZE` -> `MAX_TASK_DIRECT_RESULT_SIZE`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on a change in pull request #23415: [SPARK-26445][CORE] Use ConfigEntry for hardcoded configs for driver/executor categories.
dongjoon-hyun commented on a change in pull request #23415: [SPARK-26445][CORE] Use ConfigEntry for hardcoded configs for driver/executor categories. URL: https://github.com/apache/spark/pull/23415#discussion_r244628594 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -503,12 +503,12 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria logWarning(msg) } -val executorOptsKey = "spark.executor.extraJavaOptions" -val executorClasspathKey = "spark.executor.extraClassPath" -val driverOptsKey = "spark.driver.extraJavaOptions" -val driverClassPathKey = "spark.driver.extraClassPath" -val driverLibraryPathKey = "spark.driver.extraLibraryPath" -val sparkExecutorInstances = "spark.executor.instances" +val executorOptsKey = EXECUTOR_JAVA_OPTIONS.key +val executorClasspathKey = EXECUTOR_CLASS_PATH.key +val driverOptsKey = DRIVER_JAVA_OPTIONS.key +val driverClassPathKey = DRIVER_CLASS_PATH.key +val driverLibraryPathKey = DRIVER_LIBRARY_PATH.key +val sparkExecutorInstances = EXECUTOR_INSTANCES.key Review comment: It seems that `sparkExecutorInstances` is not used. Can we clean up unused variables in this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on a change in pull request #23415: [SPARK-26445][CORE] Use ConfigEntry for hardcoded configs for driver/executor categories.
dongjoon-hyun commented on a change in pull request #23415: [SPARK-26445][CORE] Use ConfigEntry for hardcoded configs for driver/executor categories. URL: https://github.com/apache/spark/pull/23415#discussion_r244628594 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -503,12 +503,12 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria logWarning(msg) } -val executorOptsKey = "spark.executor.extraJavaOptions" -val executorClasspathKey = "spark.executor.extraClassPath" -val driverOptsKey = "spark.driver.extraJavaOptions" -val driverClassPathKey = "spark.driver.extraClassPath" -val driverLibraryPathKey = "spark.driver.extraLibraryPath" -val sparkExecutorInstances = "spark.executor.instances" +val executorOptsKey = EXECUTOR_JAVA_OPTIONS.key +val executorClasspathKey = EXECUTOR_CLASS_PATH.key +val driverOptsKey = DRIVER_JAVA_OPTIONS.key +val driverClassPathKey = DRIVER_CLASS_PATH.key +val driverLibraryPathKey = DRIVER_LIBRARY_PATH.key +val sparkExecutorInstances = EXECUTOR_INSTANCES.key Review comment: It seems that `sparkExecutorInstances` is not used. Can we clean up unused variable in this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450710515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100602/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450710514 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450710515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100602/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450710514 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
SparkQA removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450703961 **[Test build #100602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100602/testReport)** for PR 23400 at commit [`a5dedac`](https://github.com/apache/spark/commit/a5dedacff6bcb6d0bc0fcacc7dfc2420a12ad0bb). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450710443 **[Test build #100602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100602/testReport)** for PR 23400 at commit [`a5dedac`](https://github.com/apache/spark/commit/a5dedacff6bcb6d0bc0fcacc7dfc2420a12ad0bb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
SparkQA removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450709673 **[Test build #100605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100605/testReport)** for PR 23421 at commit [`072dbbf`](https://github.com/apache/spark/commit/072dbbf1a278e518a10c7711bd07f71f690ab813). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450710395 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100605/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450710393 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
SparkQA commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450710386 **[Test build #100605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100605/testReport)** for PR 23421 at commit [`072dbbf`](https://github.com/apache/spark/commit/072dbbf1a278e518a10c7711bd07f71f690ab813). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450710393 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450710395 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100605/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on a change in pull request #23419: [SPARK-26507][CORE] Fix core tests for Java 11
dongjoon-hyun commented on a change in pull request #23419: [SPARK-26507][CORE]
Fix core tests for Java 11
URL: https://github.com/apache/spark/pull/23419#discussion_r244628340
##
File path: pom.xml
##
@@ -2113,7 +2115,7 @@
**/*Suite.java
${project.build.directory}/surefire-reports
--ea -Xmx3g -Xss4m
-XX:ReservedCodeCacheSize=${CodeCacheSize}
+-ea -Xmx6g -Xss4m
-XX:ReservedCodeCacheSize=${CodeCacheSize}
Review comment:
+1 for increasing this. I also saw multiple TimSort OOM failures even in
Apache Spark Jenkins environment(JDK8).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core
AmplabJenkins commented on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core URL: https://github.com/apache/spark/pull/23422#issuecomment-450710327 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core
AmplabJenkins removed a comment on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core URL: https://github.com/apache/spark/pull/23422#issuecomment-450710316 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core
AmplabJenkins commented on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core URL: https://github.com/apache/spark/pull/23422#issuecomment-450710316 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses
gatorsmile commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses URL: https://github.com/apache/spark/pull/20433#issuecomment-450710303 @maropu I am assuming this is blocked by https://github.com/apache/spark/pull/23259, right? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core
SparkQA commented on issue #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core URL: https://github.com/apache/spark/pull/23422#issuecomment-450710240 **[Test build #100607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100607/testReport)** for PR 23422 at commit [`49417df`](https://github.com/apache/spark/commit/49417dfe0478bda6d47e2818b221b99ef7064847). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on issue #23259: [SPARK-26215][SQL][WIP] Define reserved/non-reserved keywords based on the ANSI SQL standard
gatorsmile commented on issue #23259: [SPARK-26215][SQL][WIP] Define reserved/non-reserved keywords based on the ANSI SQL standard URL: https://github.com/apache/spark/pull/23259#issuecomment-450710217 ping @maropu Anything is blocking this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] caneGuy opened a new pull request #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core
caneGuy opened a new pull request #23422: [SPARK-26514][CORE] Support running multi tasks per cpu core URL: https://github.com/apache/spark/pull/23422 ## What changes were proposed in this pull request? Introduce `spark.cpu.tasks` to improve performance for some applications and can also improve resource utilization. ## How was this patch tested? UT This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] kiszk commented on a change in pull request #22954: [SPARK-25981][R] Enables Arrow optimization from R DataFrame to Spark DataFrame
kiszk commented on a change in pull request #22954: [SPARK-25981][R] Enables
Arrow optimization from R DataFrame to Spark DataFrame
URL: https://github.com/apache/spark/pull/22954#discussion_r244628231
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -1287,8 +1287,9 @@ object SQLConf {
val ARROW_EXECUTION_ENABLED =
buildConf("spark.sql.execution.arrow.enabled")
.doc("When true, make use of Apache Arrow for columnar data transfers.
Currently available " +
-"for use with pyspark.sql.DataFrame.toPandas, and " +
-"pyspark.sql.SparkSession.createDataFrame when its input is a Pandas
DataFrame. " +
+"for use with pyspark.sql.DataFrame.toPandas, " +
+"pyspark.sql.SparkSession.createDataFrame when its input is a Pandas
DataFrame," +
Review comment:
super nit if you will need additional commit: `... Pandas DataFrame," +` ->
`... Pandas DataFrame, " +`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com
AmplabJenkins removed a comment on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com URL: https://github.com/apache/spark/pull/23420#issuecomment-450709693 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6532/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com
AmplabJenkins commented on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com URL: https://github.com/apache/spark/pull/23420#issuecomment-450709692 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com
AmplabJenkins removed a comment on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com URL: https://github.com/apache/spark/pull/23420#issuecomment-450709692 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450709653 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com
AmplabJenkins commented on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com URL: https://github.com/apache/spark/pull/23420#issuecomment-450709693 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6532/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins removed a comment on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450709655 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6531/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
SparkQA commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450709673 **[Test build #100605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100605/testReport)** for PR 23421 at commit [`072dbbf`](https://github.com/apache/spark/commit/072dbbf1a278e518a10c7711bd07f71f690ab813). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450709653 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com
SparkQA commented on issue #23420: [SPARK-26508][Core][SQL] Address warning messages in Java reported at lgtm.com URL: https://github.com/apache/spark/pull/23420#issuecomment-450709675 **[Test build #100606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100606/testReport)** for PR 23420 at commit [`557a940`](https://github.com/apache/spark/commit/557a940d5c71b2e9119a5863399fe32a9c5eb184). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
AmplabJenkins commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450709655 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6531/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation
HyukjinKwon commented on issue #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421#issuecomment-450709553 cc @felixcheung and @shivaram This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
SparkQA removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450706630 **[Test build #4491 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4491/testReport)** for PR 23400 at commit [`a5dedac`](https://github.com/apache/spark/commit/a5dedacff6bcb6d0bc0fcacc7dfc2420a12ad0bb). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450709543 **[Test build #4491 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4491/testReport)** for PR 23400 at commit [`a5dedac`](https://github.com/apache/spark/commit/a5dedacff6bcb6d0bc0fcacc7dfc2420a12ad0bb). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon opened a new pull request #23421: [MINOR][R] Deduplicate RStudio setup documentation
HyukjinKwon opened a new pull request #23421: [MINOR][R] Deduplicate RStudio setup documentation URL: https://github.com/apache/spark/pull/23421 ## What changes were proposed in this pull request? This PR targets to deduplicate RStudio setup for SparkR. ## How was this patch tested? N/A This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450709166 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100601/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450709165 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450709166 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100601/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450709165 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
SparkQA removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450703787 **[Test build #100601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100601/testReport)** for PR 23400 at commit [`a5dedac`](https://github.com/apache/spark/commit/a5dedacff6bcb6d0bc0fcacc7dfc2420a12ad0bb). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450709144 **[Test build #100601 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100601/testReport)** for PR 23400 at commit [`a5dedac`](https://github.com/apache/spark/commit/a5dedacff6bcb6d0bc0fcacc7dfc2420a12ad0bb). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SandishKumarHN edited a comment on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle
SandishKumarHN edited a comment on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle URL: https://github.com/apache/spark/pull/23401#issuecomment-450708903 @gatorsmile that's what we are doing right now. we have taken spark source code modified with a param ```spark.executor.memory.idleGCEnabled``` with default value as false. and also proposing the same thing for the spark. As an option for users to use. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SandishKumarHN commented on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle
SandishKumarHN commented on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle URL: https://github.com/apache/spark/pull/23401#issuecomment-450708903 @gatorsmile that's what we are doing right now. we have take spark source modified with a param ```spark.executor.memory.idleGCEnabled``` with default value as false. and also proposing same thing for the spark. As an option for users to use. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] dongjoon-hyun commented on a change in pull request #23387: [SPARK-26447][SQL]Allow OrcColumnarBatchReader to return less partition columns
dongjoon-hyun commented on a change in pull request #23387:
[SPARK-26447][SQL]Allow OrcColumnarBatchReader to return less partition columns
URL: https://github.com/apache/spark/pull/23387#discussion_r244627841
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala
##
@@ -539,6 +539,25 @@ object PartitioningUtils {
}).asNullable
}
+ def requestedPartitionColumnIds(
Review comment:
If you don't mind, I prefer to revert the last commit,
https://github.com/apache/spark/pull/23387/commits/1b58df82c6ac5580f14a3b68022b88915c577646
.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle
gatorsmile commented on issue #23401: [SPARK-26513][Core] : Trigger GC on executor node idle URL: https://github.com/apache/spark/pull/23401#issuecomment-450708449 @SandishKumarHN How about manually triggering a full GC in each executor based on your workload requirements? I think that might help most of your cases. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] singhpraveen2010 removed a comment on issue #22758: [SPARK-25332][SQL] Instead of broadcast hash join , Sort merge join node is added in the plan for the join queries executed in a new spark
singhpraveen2010 removed a comment on issue #22758: [SPARK-25332][SQL] Instead of broadcast hash join ,Sort merge join node is added in the plan for the join queries executed in a new spark session/context URL: https://github.com/apache/spark/pull/22758#issuecomment-450707984 test This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] singhpraveen2010 commented on issue #22758: [SPARK-25332][SQL] Instead of broadcast hash join , Sort merge join node is added in the plan for the join queries executed in a new spark session/
singhpraveen2010 commented on issue #22758: [SPARK-25332][SQL] Instead of broadcast hash join ,Sort merge join node is added in the plan for the join queries executed in a new spark session/context URL: https://github.com/apache/spark/pull/22758#issuecomment-450707984 test This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #23391: [SPARK-26456][SQL] Cast date/timestamp to string by Date/TimestampFormatter
gatorsmile commented on a change in pull request #23391: [SPARK-26456][SQL]
Cast date/timestamp to string by Date/TimestampFormatter
URL: https://github.com/apache/spark/pull/23391#discussion_r244627486
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala
##
@@ -230,7 +235,7 @@ object PartitioningUtils {
// Once we get the string, we try to parse it and find the partition
column and value.
val maybeColumn =
parsePartitionColumn(currentPath.getName, typeInference,
userSpecifiedDataTypes,
-validatePartitionColumns, timeZone)
+validatePartitionColumns, timeZone, dateFormatter,
timestampFormatter)
Review comment:
Maybe some of you have already asked this question. Do we have any forward
compatibility and backward compatibility issue here? For example, can Spark 3.0
return the same value that is wrote by Spark 2.4? and can Spark 2.4 return the
same value that is wrote by Spark 3.0?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23414: [Spark 26449][PYSPARK] add a transform method to the Dataframe class
HyukjinKwon commented on issue #23414: [Spark 26449][PYSPARK] add a transform method to the Dataframe class URL: https://github.com/apache/spark/pull/23414#issuecomment-450706854 @chanansh, also please fix the PR title to `[SPARK-26449][PYTHON] ... ` so that it automatically links your PR to the JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450706774 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100600/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450706773 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450706774 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/100600/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
SparkQA removed a comment on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450702424 **[Test build #100600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100600/testReport)** for PR 21790 at commit [`6d3bd19`](https://github.com/apache/spark/commit/6d3bd190e6301cc2892524a0884e5eddd94e306a). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
AmplabJenkins commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450706773 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed
SparkQA commented on issue #21790: [SPARK-24544][SQL] Print actual failure cause when look up function failed URL: https://github.com/apache/spark/pull/21790#issuecomment-450706747 **[Test build #100600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100600/testReport)** for PR 21790 at commit [`6d3bd19`](https://github.com/apache/spark/commit/6d3bd190e6301cc2892524a0884e5eddd94e306a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
AmplabJenkins removed a comment on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450703993 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/6530/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType
SparkQA commented on issue #23400: [SPARK-26499] [SQL] JdbcUtils.makeGetter does not handle ByteType URL: https://github.com/apache/spark/pull/23400#issuecomment-450706630 **[Test build #4491 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4491/testReport)** for PR 23400 at commit [`a5dedac`](https://github.com/apache/spark/commit/a5dedacff6bcb6d0bc0fcacc7dfc2420a12ad0bb). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
