[spark] branch master updated (780cf3c -> 67c39a0)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 780cf3c [SPARK-37747][BUILD][FOLLOWUP] Update MapStatusesSerDeserBenchmark results add 67c39a0 [SPARK-37716][SQL] Improve error messages when a LateralJoin has non-deterministic expressions No new revisions were added by this update. Summary of changes: .../sql/catalyst/analysis/CheckAnalysis.scala | 17 ++- .../resources/sql-tests/inputs/join-lateral.sql| 10 .../sql-tests/results/join-lateral.sql.out | 56 +- 3 files changed, 81 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (67c39a0 -> 75f919e)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 67c39a0 [SPARK-37716][SQL] Improve error messages when a LateralJoin has non-deterministic expressions add 75f919e [SPARK-37732][SQL][TEST] Improve the implement of JDBCV2Suite No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/jdbc/JDBCV2Suite.scala| 59 -- 1 file changed, 22 insertions(+), 37 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (d8920ae -> 6c69262)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from d8920ae [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution add 6c69262 [SPARK-37496][SQL][FOLLOWUP] DataFrameWriter.saveAsTable should take write options in CTAS and RTAS No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/DataFrameWriter.scala | 4 +- .../sql/connector/DataSourceV2DataFrameSuite.scala | 48 +- 2 files changed, 49 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated: [SPARK-37676][SQL] Support ANSI Aggregation Function: percentile_cont
This is an automated email from the ASF dual-hosted git repository. maxgekk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 73789da [SPARK-37676][SQL] Support ANSI Aggregation Function: percentile_cont 73789da is described below commit 73789da962c9037bde21a53fb5826b10475658fe Author: Jiaan Geng AuthorDate: Mon Dec 27 16:12:43 2021 +0300 [SPARK-37676][SQL] Support ANSI Aggregation Function: percentile_cont ### What changes were proposed in this pull request? `PERCENTILE_CONT` is an ANSI aggregate functions. The mainstream database supports `percentile_cont` show below: **Postgresql** https://www.postgresql.org/docs/9.4/functions-aggregate.html **Teradata** https://docs.teradata.com/r/kmuOwjp1zEYg98JsB8fu_A/cPkFySIBORL~M938Zv07Cg **Snowflake** https://docs.snowflake.com/en/sql-reference/functions/percentile_cont.html **Oracle** https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/PERCENTILE_CONT.html#GUID-CA259452-A565-41B3-A4F4-DD74B66CEDE0 **H2** http://www.h2database.com/html/functions-aggregate.html#percentile_cont **Sybase** https://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01776.1601/doc/html/san1278453109663.html **Exasol** https://docs.exasol.com/sql_references/functions/alphabeticallistfunctions/percentile_cont.htm **RedShift** https://docs.aws.amazon.com/redshift/latest/dg/r_PERCENTILE_CONT.html **Yellowbrick** https://www.yellowbrick.com/docs/2.2/ybd_sqlref/percentile_cont.html **Mariadb** https://mariadb.com/kb/en/percentile_cont/ **Phoenix** http://phoenix.incubator.apache.org/language/functions.html#percentile_cont **Singlestore** https://docs.singlestore.com/db/v7.6/en/reference/sql-reference/window-functions/percentile_cont-and-median.html ### Why are the changes needed? `PERCENTILE_CONT` is very useful. Exposing the expression can make the migration from other systems to Spark SQL easier. ### Does this PR introduce _any_ user-facing change? 'Yes'. New feature. ### How was this patch tested? New tests. Closes #34936 from beliefer/SPARK-37676. Authored-by: Jiaan Geng Signed-off-by: Max Gekk --- docs/sql-ref-ansi-compliance.md| 2 ++ .../apache/spark/sql/catalyst/parser/SqlBase.g4| 6 .../spark/sql/catalyst/parser/AstBuilder.scala | 15 +++- .../sql/catalyst/parser/PlanParserSuite.scala | 24 - .../test/resources/sql-tests/inputs/group-by.sql | 18 +- .../resources/sql-tests/results/group-by.sql.out | 41 +- 6 files changed, 102 insertions(+), 4 deletions(-) diff --git a/docs/sql-ref-ansi-compliance.md b/docs/sql-ref-ansi-compliance.md index 7b5bde4..1b4a778 100644 --- a/docs/sql-ref-ansi-compliance.md +++ b/docs/sql-ref-ansi-compliance.md @@ -494,6 +494,7 @@ Below is a list of all the keywords in Spark SQL. |PARTITIONED|non-reserved|non-reserved|non-reserved| |PARTITIONS|non-reserved|non-reserved|non-reserved| |PERCENT|non-reserved|non-reserved|non-reserved| +|PERCENTILE_CONT|reserved|non-reserved|non-reserved| |PIVOT|non-reserved|non-reserved|non-reserved| |PLACING|non-reserved|non-reserved|non-reserved| |POSITION|non-reserved|non-reserved|reserved| @@ -594,5 +595,6 @@ Below is a list of all the keywords in Spark SQL. |WHERE|reserved|non-reserved|reserved| |WINDOW|non-reserved|non-reserved|reserved| |WITH|reserved|non-reserved|reserved| +|WITHIN|reserved|non-reserved|reserved| |YEAR|non-reserved|non-reserved|non-reserved| |ZONE|non-reserved|non-reserved|non-reserved| diff --git a/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 b/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 index 6511489..5037520 100644 --- a/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 +++ b/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 @@ -888,6 +888,8 @@ primaryExpression FROM srcStr=valueExpression ')' #trim | OVERLAY '(' input=valueExpression PLACING replace=valueExpression FROM position=valueExpression (FOR length=valueExpression)? ')' #overlay +| PERCENTILE_CONT '(' percentage=valueExpression ')' + WITHIN GROUP '(' ORDER BY sortItem ')' #percentile ; constant @@ -1475,6 +1477,7 @@ nonReserved | PARTITION | PARTITIONED | PARTITIONS +| PERCENTILE_CONT | PERCENTLIT | PIVOT | PLACING @@ -1570,6 +1573,7 @@ nonReserved | WHERE | WINDOW | WITH +| WITHIN | YEAR | ZONE //--DEFAULT-NON-RESERVED-END @@ -1747,6 +1751,7 @
[spark] branch master updated (73789da -> f4d6a0f)
This is an automated email from the ASF dual-hosted git repository. viirya pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 73789da [SPARK-37676][SQL] Support ANSI Aggregation Function: percentile_cont add f4d6a0f [SPARK-37746][CORE] log4j2-defaults.properties is not working since log4j 2 is always initialized by default No new revisions were added by this update. Summary of changes: .../org/apache/spark/log4j2-defaults.properties | 2 +- .../main/scala/org/apache/spark/internal/Logging.scala | 17 +++-- 2 files changed, 12 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated: [SPARK-37755][CORE] Optimize RocksDB KVStore configurations
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 285d8dd [SPARK-37755][CORE] Optimize RocksDB KVStore configurations 285d8dd is described below commit 285d8dd9f0b8ee8b85603b4cc0f656c452bff629 Author: Dongjoon Hyun AuthorDate: Mon Dec 27 17:30:22 2021 -0800 [SPARK-37755][CORE] Optimize RocksDB KVStore configurations ### What changes were proposed in this pull request? This PR aims to optimize RocksDB KVStore configurations to improve the performance up to 4 times. ### Why are the changes needed? Currently, `RocksDB` KVStore performance is not optimized. **REFERENCE (LevelDB) ** ``` [INFO] Running org.apache.spark.util.kvstore.LevelDBBenchmark count meanmin max 95th dbClose 4 0.802 0.777 0.824 0.824 dbCreation 4 71.029 2.528 278.049 278.049 naturalIndexCreateIterator 10240.006 0.002 1.559 0.007 naturalIndexDescendingCreateIterator10240.003 0.003 0.043 0.004 naturalIndexDescendingIteration 10240.007 0.005 0.053 0.008 naturalIndexIteration 10240.007 0.005 0.080 0.009 randomDeleteIndexed 10240.020 0.017 0.148 0.024 randomDeletesNoIndex10240.025 0.008 14.739 0.013 randomUpdatesIndexed10240.079 0.033 23.527 0.075 randomUpdatesNoIndex10240.078 0.072 0.584 0.082 randomWritesIndexed 10240.121 0.043 42.030 0.141 randomWritesNoIndex 10240.086 0.059 1.204 0.090 refIndexCreateIterator 10240.003 0.002 0.033 0.004 refIndexDescendingCreateIterator10240.002 0.002 0.018 0.003 refIndexDescendingIteration 10240.007 0.006 0.045 0.008 refIndexIteration 10240.009 0.006 0.261 0.010 sequentialDeleteIndexed 10240.019 0.016 0.090 0.024 sequentialDeleteNoIndex 10240.013 0.010 0.083 0.014 sequentialUpdatesIndexed10240.031 0.027 0.560 0.035 sequentialUpdatesNoIndex10240.028 0.024 0.618 0.032 sequentialWritesIndexed 10240.037 0.031 1.301 0.041 sequentialWritesNoIndex 10240.038 0.026 1.699 0.065 ``` **AS-IS (RocksDB)** ``` [INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark count meanmin max 95th dbClose 4 0.621 0.455 0.830 0.830 dbCreation 4 93.812 4.566 365.368 365.368 naturalIndexCreateIterator 10240.005 0.002 1.349 0.007 naturalIndexDescendingCreateIterator10240.006 0.005 0.054 0.007 naturalIndexDescendingIteration 10240.007 0.005 0.034 0.009 naturalIndexIteration 10240.007 0.005 0.062 0.009 randomDeleteIndexed 10240.138 0.079 0.636 0.186 randomDeletesNoIndex10240.104 0.058 0.223 0.143 randomUpdatesIndexed10240.203 0.145 23.849 0.243 randomUpdatesNoIndex10240.161 0.122 0.509 0.196 randomWritesIndexed 10240.239 0.149 42.780 0.279 randomWritesNoIndex 10240.143 0.080 1.593 0.221 refIndexCreateIterator 10240.003 0.003 0.032 0.005 refIndexDescendingCreateIterator10240.003 0.003 0.028 0.005 refIndexDescendingIteration 10240.008 0.006 0.038 0.009 refIndexIteration 10240.009 0.007 0.265 0.011 sequentialDeleteIndexed 10240.122 0.106 0.251 0.150 sequentialDeleteNoIndex 10240.101 0.059 0.315 0.121 sequentialUpdatesIndexed10240.147 0.109 0.812 0.197 sequentialUpdatesNoIndex10240.143 0.112 0.896 0.184 sequentialWritesIndexed 10240.170 0.130 1.715 0.235 sequentialWritesNoIndex 1024
[spark] branch master updated (285d8dd -> 371e307)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 285d8dd [SPARK-37755][CORE] Optimize RocksDB KVStore configurations add 371e307 [SPARK-37730][PYTHON] Replace use of MPLPlot._add_legend_handle with MPLPlot._append_legend_handles_labels No new revisions were added by this update. Summary of changes: python/pyspark/pandas/plot/matplotlib.py | 22 -- 1 file changed, 20 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (371e307 -> 80dca37)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 371e307 [SPARK-37730][PYTHON] Replace use of MPLPlot._add_legend_handle with MPLPlot._append_legend_handles_labels add 80dca37 [SPARK-37760][BUILD] Upgrade SBT to 1.6.0 No new revisions were added by this update. Summary of changes: dev/appveyor-install-dependencies.ps1 | 2 +- project/build.properties | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (80dca37 -> 78899e3)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 80dca37 [SPARK-37760][BUILD] Upgrade SBT to 1.6.0 add 78899e3 [SPARK-37756][PYTHON] Enable matplotlib test for pandas API on Spark No new revisions were added by this update. Summary of changes: .github/workflows/build_and_test.yml | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated: [SPARK-37728][SQL] Reading nested columns with ORC vectorized reader can cause ArrayIndexOutOfBoundsException
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d245aef [SPARK-37728][SQL] Reading nested columns with ORC vectorized reader can cause ArrayIndexOutOfBoundsException d245aef is described below commit d245aefe87cc974fe7fc37587d09bec12c1bf911 Author: Yimin AuthorDate: Mon Dec 27 22:09:51 2021 -0800 [SPARK-37728][SQL] Reading nested columns with ORC vectorized reader can cause ArrayIndexOutOfBoundsException ### What changes were proposed in this pull request? When an OrcColumnarBatchReader is created, method initBatch will be called only once. In method initBatch: `orcVectorWrappers[i] = OrcColumnVectorUtils.toOrcColumnVector(dt, wrap.batch().cols[colId]);` When the second argument of toOrcColumnVector is a ListColumnVector/MapColumnVector, orcVectorWrappers[i] is initialized with the ListColumnVector or MapColumnVector's offsets and lengths. However, when method nextBatch of OrcColumnarBatchReader is called, method ensureSize of ColumnVector (and its subclasses, like MultiValuedColumnVector) could be called, then the ListColumnVector/MapColumnVector's offsets and lengths could refer to new array objects. This could result in the ArrayIndexOutOfBoundsException. This PR makes OrcArrayColumnVector.getArray and OrcMapColumnVector.getMap always get offsets and lengths from the underlying ColumnVector, which can resolve this issue. ### Why are the changes needed? Bugfix ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass the CIs with the newly added test case. Closes #35002 from yym1995/fix-nested. Lead-authored-by: Yimin Co-authored-by: Yimin <26797163+yym1...@users.noreply.github.com> Signed-off-by: Dongjoon Hyun --- .../datasources/orc/OrcArrayColumnVector.java | 13 + .../execution/datasources/orc/OrcColumnVector.java | 2 +- .../datasources/orc/OrcColumnVectorUtils.java | 6 ++ .../datasources/orc/OrcMapColumnVector.java| 13 + .../execution/datasources/orc/OrcQuerySuite.scala | 22 ++ 5 files changed, 35 insertions(+), 21 deletions(-) diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcArrayColumnVector.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcArrayColumnVector.java index 6e13e97..b0c818f 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcArrayColumnVector.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcArrayColumnVector.java @@ -18,6 +18,7 @@ package org.apache.spark.sql.execution.datasources.orc; import org.apache.hadoop.hive.ql.exec.vector.ColumnVector; +import org.apache.hadoop.hive.ql.exec.vector.ListColumnVector; import org.apache.spark.sql.types.ArrayType; import org.apache.spark.sql.types.DataType; @@ -31,26 +32,22 @@ import org.apache.spark.unsafe.types.UTF8String; */ public class OrcArrayColumnVector extends OrcColumnVector { private final OrcColumnVector data; - private final long[] offsets; - private final long[] lengths; OrcArrayColumnVector( DataType type, ColumnVector vector, - OrcColumnVector data, - long[] offsets, - long[] lengths) { + OrcColumnVector data) { super(type, vector); this.data = data; -this.offsets = offsets; -this.lengths = lengths; } @Override public ColumnarArray getArray(int rowId) { -return new ColumnarArray(data, (int) offsets[rowId], (int) lengths[rowId]); +int offsets = (int) ((ListColumnVector) baseData).offsets[rowId]; +int lengths = (int) ((ListColumnVector) baseData).lengths[rowId]; +return new ColumnarArray(data, offsets, lengths); } @Override diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java index 0becd25..7fe1b30 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java @@ -29,7 +29,7 @@ import org.apache.spark.sql.vectorized.ColumnarBatch; * this column vector is used to adapt Hive ColumnVector with Spark ColumnarVector. */ public abstract class OrcColumnVector extends org.apache.spark.sql.vectorized.ColumnVector { - private final ColumnVector baseData; + protected final ColumnVector baseData; private int batchSize; OrcColumnVector(DataType type, ColumnVector vector) { diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/da
[spark] branch master updated: [SPARK-37715][SQL][BUILD][TESTS] Remove `ojdbc6` dependency and update docker-integration test docs
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 624d3b7 [SPARK-37715][SQL][BUILD][TESTS] Remove `ojdbc6` dependency and update docker-integration test docs 624d3b7 is described below commit 624d3b7c49abbf6f7bd82fa519b9270e56c660ee Author: Danny Guinther AuthorDate: Mon Dec 27 22:39:50 2021 -0800 [SPARK-37715][SQL][BUILD][TESTS] Remove `ojdbc6` dependency and update docker-integration test docs ### What changes were proposed in this pull request? This PR aims to remove unused `ojdbc6` dependency because we are using `ojdbc8` in `pom.xml`. In addition, this updates two image names in documentation. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass the GitHub Action Docker test. Closes #34979 from tdg5/just-the-docker-stuff. Authored-by: Danny Guinther Signed-off-by: Dongjoon Hyun --- .../scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala | 3 ++- .../org/apache/spark/sql/jdbc/v2/MsSqlServerIntegrationSuite.scala | 3 ++- project/SparkBuild.scala | 3 +-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala b/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala index 78015d8..5992253 100644 --- a/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala +++ b/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala @@ -28,7 +28,8 @@ import org.apache.spark.tags.DockerTest /** * To run this test suite for a specific version (e.g., 2019-CU13-ubuntu-20.04): * {{{ - * ENABLE_DOCKER_INTEGRATION_TESTS=1 MSSQLSERVER_DOCKER_IMAGE_NAME=2019-CU13-ubuntu-20.04 + * ENABLE_DOCKER_INTEGRATION_TESTS=1 + * MSSQLSERVER_DOCKER_IMAGE_NAME=mcr.microsoft.com/mssql/server:2019-CU13-ubuntu-20.04 * ./build/sbt -Pdocker-integration-tests * "testOnly org.apache.spark.sql.jdbc.MsSqlServerIntegrationSuite" * }}} diff --git a/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MsSqlServerIntegrationSuite.scala b/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MsSqlServerIntegrationSuite.scala index 77b7742..75446fb 100644 --- a/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MsSqlServerIntegrationSuite.scala +++ b/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MsSqlServerIntegrationSuite.scala @@ -31,7 +31,8 @@ import org.apache.spark.tags.DockerTest /** * To run this test suite for a specific version (e.g., 2019-CU13-ubuntu-20.04): * {{{ - * ENABLE_DOCKER_INTEGRATION_TESTS=1 MSSQLSERVER_DOCKER_IMAGE_NAME=2019-CU13-ubuntu-20.04 + * ENABLE_DOCKER_INTEGRATION_TESTS=1 + * MSSQLSERVER_DOCKER_IMAGE_NAME=mcr.microsoft.com/mssql/server:2019-CU13-ubuntu-20.04 * ./build/sbt -Pdocker-integration-tests "testOnly *v2*MsSqlServerIntegrationSuite" * }}} */ diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index 2f21561..579822a 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -599,8 +599,7 @@ object DockerIntegrationTests { // This serves to override the override specified in DependencyOverrides: lazy val settings = Seq( dependencyOverrides += "com.google.guava" % "guava" % "18.0", -resolvers += "DB2" at "https://app.camunda.com/nexus/content/repositories/public/";, -libraryDependencies += "com.oracle" % "ojdbc6" % "11.2.0.1.0" from "https://app.camunda.com/nexus/content/repositories/public/com/oracle/ojdbc6/11.2.0.1.0/ojdbc6-11.2.0.1.0.jar"; // scalastyle:ignore +resolvers += "DB2" at "https://app.camunda.com/nexus/content/repositories/public/"; ) } - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated: [SPARK-37763][BUILD] Upgrade jackson to 2.13.1
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e653b8c [SPARK-37763][BUILD] Upgrade jackson to 2.13.1 e653b8c is described below commit e653b8cd89ea13c22823885c71cfb26514c72a03 Author: William Hyun AuthorDate: Mon Dec 27 23:28:39 2021 -0800 [SPARK-37763][BUILD] Upgrade jackson to 2.13.1 ### What changes were proposed in this pull request? This PR aims to upgrade jackson to 2.13.1. ### Why are the changes needed? This release has the following fix. - https://github.com/FasterXML/jackson-core/issues/721 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. Closes #35037 from williamhyun/jackson2131. Authored-by: William Hyun Signed-off-by: Dongjoon Hyun --- dev/deps/spark-deps-hadoop-2-hive-2.3 | 12 ++-- dev/deps/spark-deps-hadoop-3-hive-2.3 | 12 ++-- pom.xml | 2 +- 3 files changed, 13 insertions(+), 13 deletions(-) diff --git a/dev/deps/spark-deps-hadoop-2-hive-2.3 b/dev/deps/spark-deps-hadoop-2-hive-2.3 index e38213f..9ad5f18 100644 --- a/dev/deps/spark-deps-hadoop-2-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-2-hive-2.3 @@ -111,16 +111,16 @@ httpclient/4.5.13//httpclient-4.5.13.jar httpcore/4.4.14//httpcore-4.4.14.jar istack-commons-runtime/3.0.8//istack-commons-runtime-3.0.8.jar ivy/2.5.0//ivy-2.5.0.jar -jackson-annotations/2.13.0//jackson-annotations-2.13.0.jar +jackson-annotations/2.13.1//jackson-annotations-2.13.1.jar jackson-core-asl/1.9.13//jackson-core-asl-1.9.13.jar -jackson-core/2.13.0//jackson-core-2.13.0.jar -jackson-databind/2.13.0//jackson-databind-2.13.0.jar -jackson-dataformat-cbor/2.13.0//jackson-dataformat-cbor-2.13.0.jar -jackson-dataformat-yaml/2.13.0//jackson-dataformat-yaml-2.13.0.jar +jackson-core/2.13.1//jackson-core-2.13.1.jar +jackson-databind/2.13.1//jackson-databind-2.13.1.jar +jackson-dataformat-cbor/2.13.1//jackson-dataformat-cbor-2.13.1.jar +jackson-dataformat-yaml/2.13.1//jackson-dataformat-yaml-2.13.1.jar jackson-datatype-jsr310/2.13.0//jackson-datatype-jsr310-2.13.0.jar jackson-jaxrs/1.9.13//jackson-jaxrs-1.9.13.jar jackson-mapper-asl/1.9.13//jackson-mapper-asl-1.9.13.jar -jackson-module-scala_2.12/2.13.0//jackson-module-scala_2.12-2.13.0.jar +jackson-module-scala_2.12/2.13.1//jackson-module-scala_2.12-2.13.1.jar jackson-xc/1.9.13//jackson-xc-1.9.13.jar jakarta.annotation-api/1.3.5//jakarta.annotation-api-1.3.5.jar jakarta.inject/2.6.1//jakarta.inject-2.6.1.jar diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3 index e0286e6..a482b33 100644 --- a/dev/deps/spark-deps-hadoop-3-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-3-hive-2.3 @@ -103,15 +103,15 @@ httpclient/4.5.13//httpclient-4.5.13.jar httpcore/4.4.14//httpcore-4.4.14.jar istack-commons-runtime/3.0.8//istack-commons-runtime-3.0.8.jar ivy/2.5.0//ivy-2.5.0.jar -jackson-annotations/2.13.0//jackson-annotations-2.13.0.jar +jackson-annotations/2.13.1//jackson-annotations-2.13.1.jar jackson-core-asl/1.9.13//jackson-core-asl-1.9.13.jar -jackson-core/2.13.0//jackson-core-2.13.0.jar -jackson-databind/2.13.0//jackson-databind-2.13.0.jar -jackson-dataformat-cbor/2.13.0//jackson-dataformat-cbor-2.13.0.jar -jackson-dataformat-yaml/2.13.0//jackson-dataformat-yaml-2.13.0.jar +jackson-core/2.13.1//jackson-core-2.13.1.jar +jackson-databind/2.13.1//jackson-databind-2.13.1.jar +jackson-dataformat-cbor/2.13.1//jackson-dataformat-cbor-2.13.1.jar +jackson-dataformat-yaml/2.13.1//jackson-dataformat-yaml-2.13.1.jar jackson-datatype-jsr310/2.13.0//jackson-datatype-jsr310-2.13.0.jar jackson-mapper-asl/1.9.13//jackson-mapper-asl-1.9.13.jar -jackson-module-scala_2.12/2.13.0//jackson-module-scala_2.12-2.13.0.jar +jackson-module-scala_2.12/2.13.1//jackson-module-scala_2.12-2.13.1.jar jakarta.annotation-api/1.3.5//jakarta.annotation-api-1.3.5.jar jakarta.inject/2.6.1//jakarta.inject-2.6.1.jar jakarta.servlet-api/4.0.3//jakarta.servlet-api-4.0.3.jar diff --git a/pom.xml b/pom.xml index 0facb65..7314eb1 100644 --- a/pom.xml +++ b/pom.xml @@ -167,7 +167,7 @@ true 1.9.13 -2.13.0 +2.13.1 1.1.8.4 1.1.2 2.2.1 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org