commits
Thread
Date
Earlier messages
Later messages
Messages by Thread
(spark) branch master updated (ca0001345d0b -> 7f1eadcd9cbb)
gurwls223
(spark) branch branch-3.5 updated: [SPARK-47636][K8S][3.5] Use Java `17` instead of `17-jre` image in K8s Dockerfile
dongjoon
(spark) branch master updated (c832e2ac1d04 -> ca0001345d0b)
dongjoon
(spark) branch master updated: [SPARK-47492][SQL] Widen whitespace rules in lexer
gengliang
(spark) branch master updated (1623b2d513d2 -> d8dc0c3e5e8a)
dongjoon
(spark) branch master updated (2e4f2b0d307f -> 1623b2d513d2)
dongjoon
(spark) branch master updated: [SPARK-47475][CORE][K8S] Support `spark.kubernetes.jars.avoidDownloadSchemes` for K8s Cluster Mode
dongjoon
(spark) branch master updated: [MINOR][CORE] Replace `get+getOrElse` with `getOrElse` with default value in `StreamingQueryException`
dongjoon
(spark) branch master updated (8c4d6764674f -> 4b58a631fea9)
dongjoon
(spark) branch master updated: [SPARK-47559][SQL] Codegen Support for variant `parse_json`
wenchen
(spark) branch master updated: [SPARK-47621][PYTHON][DOCS] Refine docstring of `try_sum`, `try_avg`, `avg`, `sum`, `mean`
ruifengz
(spark) branch master updated: [MINOR][PYTHON][TESTS] Remove redundant parity tests
ruifengz
(spark) branch master updated: [SPARK-47614][CORE][DOC] Update some outdated comments about `JavaModuleOptions`
yao
(spark) branch master updated: [SPARK-47363][SS] Initial State without state reader implementation for State API v2
kabhwan
(spark) branch master updated: [SPARK-47619][PYTHON][DOCS] Refine docstring of `to_json/from_json`
gurwls223
(spark) branch master updated: [SPARK-47620][PYTHON][CONNECT] Add a helper function to sort columns
ruifengz
(spark) branch master updated: [SPARK-47546][SQL] Improve validation when reading Variant from Parquet
wenchen
(spark) branch master updated: [SPARK-47543][CONNECT][PYTHON] Inferring `dict` as `MapType` from Pandas DataFrame to allow DataFrame creation
gurwls223
(spark) branch master updated: [SPARK-47107][SS][PYTHON] Implement partition reader for python streaming data source
kabhwan
(spark) branch master updated (8d1539f7bb23 -> 1b3e5e71f7f9)
maxgekk
(spark) branch master updated: [SPARK-47616][SQL] Add User Document for Mapping Spark SQL Data Types from MySQL
dongjoon
(spark) branch master updated (87449c3f1d65 -> d10dbaa31a44)
maxgekk
(spark) branch master updated: [SPARK-47563][SQL] Add map normalization on creation
wenchen
(spark) branch master updated: [SPARK-46575][SQL][FOLLOWUP] Correct @since annotation for HiveThriftServer2.startWithContext(SQLContext, exitonError)
yao
(spark) branch master updated (b540cc538614 -> d57164a5cab9)
wenchen
(spark) branch master updated: [SPARK-47611][SQL] Cleanup dead code in MySQLDialect.getCatalystType
dongjoon
(spark) branch master updated (f9eb3f3c13bf -> a600c0ea3159)
dongjoon
(spark) branch master updated: [SPARK-46575][SQL][FOLLOWUP] Add back `HiveThriftServer2.startWithContext(SQLContext)` method for compatibility
wenchen
(spark) branch master updated: [SPARK-47610][CORE] Always set `io.netty.tryReflectionSetAccessible=true`
yangjie01
(spark) branch master updated: [SPARK-42040][SQL] SPJ: Introduce a new API for V2 input partition to report partition statistics
sunchao
(spark) branch master updated (88b29c5076d4 -> 9db527ec5adc)
ruifengz
(spark) branch master updated: [SPARK-47570][SS] Integrate range scan encoder changes with timer implementation
kabhwan
(spark) branch master updated: [SPARK-47273][SS][PYTHON] implement Python data stream writer interface
kabhwan
(spark) branch master updated: [SPARK-47498][TESTS][CORE] Refine some GPU fraction calculation tests
wuyi
(spark) branch branch-3.5 updated: [SPARK-47561][SQL] Fix analyzer rule order issues about Alias
wenchen
(spark) branch master updated (fd4b8e89f3a0 -> 7d87a94dd77f)
dongjoon
(spark) branch master updated (e00eace41a63 -> fd4b8e89f3a0)
dongjoon
(spark) branch master updated: [SPARK-47561][SQL] Fix analyzer rule order issues about Alias
dongjoon
(spark) branch master updated: [SPARK-47544][PYTHON] SparkSession builder method is incompatible with visual studio code intellisense
dongjoon
(spark) branch master updated: [SPARK-47557][SQL][TEST] Audit MySQL ENUM/SET Types
dongjoon
(spark) branch master updated: [SPARK-47367][PYTHON][CONNECT][TESTS][FOLLOW-UP] Recover the test case for the number of partitions
dongjoon
(spark) branch master updated: [SPARK-47431][SQL] Add session level default Collation
wenchen
(spark) branch master updated: [SPARK-47469][SS][TESTS] Add `Trigger.AvailableNow` tests for `transformWithState` operator
kabhwan
(spark) branch master updated: [SPARK-47560][PYTHON][CONNECT] Avoid RPC to validate column name with cached schema
ruifengz
(spark) branch master updated (a3aa08697686 -> a127df47f7a2)
gurwls223
(spark) branch master updated (ff38378d7e42 -> a3aa08697686)
maxgekk
(spark) branch master updated: [SPARK-47372][SS] Add support for range scan based key state encoder for use with state store provider
kabhwan
(spark) branch master updated (acf13d6515f7 -> 31db27d193fb)
gurwls223
(spark) branch master updated (0054a12a013f -> acf13d6515f7)
wenchen
(spark) branch master updated (2a8bb5cdd3a5 -> 0054a12a013f)
gurwls223
(spark) branch master updated (0340f805fdd2 -> 2a8bb5cdd3a5)
gurwls223
(spark) branch master updated: [SPARK-47497][SQL][FOLLOWUP] Add a UT for `nested structure` for the function `to_csv`
wenchen
(spark) branch master updated: [SPARK-47549][BUILD] Remove Spark 3.0~3.2 `pyspark/version.py` workaround from release scripts
gurwls223
(spark) branch master updated: [SPARK-47552][CORE] Set `spark.hadoop.fs.s3a.connection.establish.timeout` to 30s if missing
dongjoon
(spark) branch master updated: [SPARK-47550][K8S][BUILD] Update `kubernetes-client` to 6.11.0
dongjoon
(spark) branch master updated: [SPARK-47548][BUILD] Remove unused `commons-beanutils` dependency
dongjoon
(spark) branch master updated (b2f6474848fc -> d7db869de609)
maxgekk
(spark) branch master updated (1b55fd3ee107 -> b2f6474848fc)
wenchen
(spark) branch branch-3.4 updated (77fd58bf8d52 -> 5e7600eab833)
dongjoon
(spark) branch branch-3.4 updated: [SPARK-47537][SQL][3.4] Fix error data type mapping on MySQL Connector/J
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47537][SQL][3.5] Fix error data type mapping on MySQL Connector/J
dongjoon
(spark) branch master updated: [SPARK-47539][SQL][FOLLOWUP] Fix UT about `variant`
maxgekk
(spark-website) branch asf-site updated: Update the organization for beliefer (#510)
ruifengz
[PR] Update the organization for beliefer [spark-website]
via GitHub
Re: [PR] Update the organization for beliefer [spark-website]
via GitHub
Re: [PR] Update the organization for beliefer [spark-website]
via GitHub
(spark) branch master updated: [SPARK-47539][SQL] Make the return value of method `castToString` be `Any => UTF8String`
gurwls223
(spark) branch master updated: [SPARK-47451][SQL] Support to_json(variant).
wenchen
(spark) branch master updated: [SPARK-47538][BUILD] Remove `commons-logging` dependency
dongjoon
(spark) branch master updated: [SPARK-47537][SQL] Fix error data type mapping on MySQL Connector/J
dongjoon
(spark) branch master updated: [SPARK-47506][SQL] Add support to all file source formats for collated data types
maxgekk
(spark) branch master updated (b34a44f175aa -> d8d119a21e07)
dongjoon
(spark) branch master updated (99fb84b7ad27 -> b34a44f175aa)
dongjoon
(spark) branch master updated: [SPARK-47534][SQL] Move `o.a.s.variant` to `o.a.s.types.variant`
dongjoon
(spark) branch master updated: [SPARK-47516][INFRA] Move `remove unused installation package logic` from `each test job` to `create the docker image`
gurwls223
(spark) branch master updated: [SPARK-44708][PYTHON] Migrate test_reset_index assert_eq to use assertDataFrameEqual
gurwls223
(spark) branch master updated: [MINOR][PYTHON][DOCS] Fix a `pandas_udf` example
gurwls223
(spark) branch master updated: [SPARK-47533][BUILD] Migrate scalafmt dialect to `scala213`
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47503][SQL][3.5] Make makeDotNode escape graph node name always
dongjoon
(spark) branch master updated: [SPARK-47528][SQL] Add UserDefinedType support to DataTypeUtils.canWrite
dongjoon
(spark) branch master updated: [SPARK-47526][BUILD] Upgrade `netty` to 4.1.108.Final and `netty-tcnative` to 2.0.65.Final
dongjoon
(spark) branch master updated: [SPARK-47503][SQL] Make `makeDotNode` escape graph node name always
dongjoon
(spark) branch master updated: [SPARK-47497][SQL] Make `to_csv` support the output of `array/struct/map/binary` as pretty strings
dongjoon
(spark) branch master updated: [SPARK-47531][BUILD] Upgrade `Arrow` to 15.0.2
dongjoon
(spark) branch master updated (9d6b9f7305f2 -> 11f5d3fa10b3)
dongjoon
(spark) branch master updated: [SPARK-47529][DOCS] Use hadoop 3.4.0 in some docs
dongjoon
(spark) branch master updated (b9335b90280a -> c29d132aeb5d)
dongjoon
(spark) branch master updated (39500a315166 -> b9335b90280a)
dongjoon
(spark) branch master updated: [SPARK-47522][SQL][FOLLOWUP] Add float(p) values for MySQLIntegrationSuite
dongjoon
(spark) branch master updated (245669053a34 -> 36126a5c1821)
dongjoon
(spark) branch master updated (93f98c0a61dd -> 245669053a34)
dongjoon
(spark) branch branch-3.4 updated: [SPARK-47521][CORE] Use `Utils.tryWithResource` during reading shuffle data from external storage
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47521][CORE] Use `Utils.tryWithResource` during reading shuffle data from external storage
dongjoon
(spark) branch master updated: [SPARK-47523][SQL] Replace deprecated `JsonParser#getCurrentName` with `JsonParser#currentName`
dongjoon
(spark) branch master updated (32dfdd305aec -> d1be4fb61368)
dongjoon
(spark) branch master updated (d7be50f122ed -> 32dfdd305aec)
yao
(spark) branch master updated: [MINOR][DOCS] Fix typo in spark connect overview
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47440][SQL] Fix pushing unsupported syntax to MsSqlServer
yao
(spark) branch master updated: [SPARK-47440][SQL] Fix pushing unsupported syntax to MsSqlServer
yao
(spark) branch master updated: [SPARK-47515][SQL] Save TimestampNTZType as DATETIME in MySQL
yao
(spark) branch master updated (105f008037e5 -> c486ed813ab5)
yao
(spark) branch master updated (c94090e13139 -> 105f008037e5)
gurwls223
(spark) branch master updated (8cba15ed30ea -> c94090e13139)
maxgekk
(spark) branch master updated: [SPARK-47483][SQL] Add support for aggregation and join operations on arrays of collated strings
maxgekk
(spark) branch master updated (aea13fca5d57 -> ca44489f4585)
dongjoon
(spark) branch master updated: [SPARK-47500][PYTHON][CONNECT] Factor column name handling out of `plan.py`
gurwls223
(spark) branch branch-3.5 updated: [SPARK-47462][SQL][FOLLOWUP][3.5] Add migration guide for TINYINT mapping changes
dongjoon
(spark) branch master updated (0ef7b771b33d -> 47bce8ececa8)
dongjoon
(spark) branch master updated (5042263f8668 -> 0ef7b771b33d)
dongjoon
(spark) branch master updated: [SPARK-47479][SQL] Optimize cannot write data to relations with multiple paths error log
maxgekk
(spark) branch master updated: [SPARK-47514][SQL][TESTS] Add a test coverage for createTable method (partitioned-table) in CatalogSuite
dongjoon
(spark) branch master updated (057acf986d78 -> 7b575d137a1a)
wenchen
(spark) branch master updated: [SPARK-47512][SS] Tag operation type used with RocksDB state store instance lock acquisition/release
kabhwan
(spark) branch master updated (6a27789ad7d5 -> b4c09221b2e0)
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47398][SQL] Extract a trait for InMemoryTableScanExec to allow for extending functionality
tgraves
(spark) branch master updated: [SPARK-47398][SQL] Extract a trait for InMemoryTableScanExec to allow for extending functionality
tgraves
(spark) branch branch-3.5 updated: [SPARK-47507][BUILD][3.5] Upgrade ORC to 1.9.3
dongjoon
(spark) branch branch-3.4 updated: [SPARK-47505][INFRA][3.4] Fix `Pyspark-errors` test jobs for branch-3.4
dongjoon
(spark) branch master updated: [SPARK-47487][SQL] Simplify code in AnsiTypeCoercion
dongjoon
(spark) branch master updated: [SPARK-47501][SQL] Add convertDateToDate like the existing convertTimestampToTimestamp for JdbcDialect
dongjoon
(spark) branch master updated: [SPARK-47496][SQL] Java SPI Support for dynamic JDBC dialect registering
yao
(spark) branch master updated: [SPARK-45393][BUILD][FOLLOWUP] Update IsolatedClientLoader fallback Hadoop version to 3.4.0
dongjoon
(spark) branch master updated: [SPARK-41888][PYTHON][CONNECT][TESTS] Enable doctest for `DataFrame.observe`
dongjoon
(spark) branch master updated: [SPARK-47474][CORE] Revert SPARK-47461 and add some comments
yangjie01
(spark) branch master updated (bb0867f54d43 -> 5d3845f2942a)
yangjie01
(spark) branch branch-3.4 updated: [MINOR][CORE] Fix a comment typo `slf4j-to-jul` to `jul-to-slf4j`
dongjoon
(spark) branch branch-3.5 updated: [MINOR][CORE] Fix a comment typo `slf4j-to-jul` to `jul-to-slf4j`
dongjoon
(spark) branch master updated: [MINOR][CORE] Fix a comment typo `slf4j-to-jul` to `jul-to-slf4j`
dongjoon
(spark) branch master updated: Revert "[SPARK-47007][SQL][PYTHON][R][CONNECT] Add the `map_sort` function"
wenchen
(spark) branch branch-3.4 updated: [SPARK-47494][DOC] Add migration doc for the behavior change of Parquet timestamp inference since Spark 3.3
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47494][DOC] Add migration doc for the behavior change of Parquet timestamp inference since Spark 3.3
dongjoon
(spark) branch master updated (b8e7d99d417a -> 11247d804cd3)
dongjoon
(spark) branch master updated: [SPARK-47490][SS] Fix RocksDB Logger constructor use to avoid deprecation warning
dongjoon
(spark) branch master updated: [SPARK-47486][CONNECT] Remove unused private `ArrowDeserializers.getString` method
dongjoon
(spark) branch dependabot/pip/dev/black-24.3.0 deleted (was 39b210e90780)
github-bot
(spark) branch dependabot/pip/dev/black-24.3.0 created (now 39b210e90780)
github-bot
(spark) branch master updated: [SPARK-45393][BUILD] Upgrade Hadoop to 3.4.0
dongjoon
(spark) branch master updated: [SPARK-47462][SQL] Align mappings of other unsigned numeric types with TINYINT in MySQLDialect
dongjoon
(spark) branch master updated (76260807eef2 -> 3a3477b0f156)
maxgekk
(spark) branch master updated: [SPARK-46990][SQL] Fix loading empty Avro files emitted by event-hubs
wenchen
(spark) branch master updated (a3c04ec11456 -> 8762e256d164)
wenchen
(spark) branch branch-3.5 updated: [SPARK-47481][INFRA][3.5] Fix Python linter
dongjoon
(spark) branch branch-3.4 updated: [SPARK-47481][INFRA][3.4] Pin `matplotlib<3.3.0` to fix Python linter failure
dongjoon
(spark) branch master updated: [SPARK-47478][SQL][TESTS] Improve test coverage for mysql bool synonyms
gurwls223
(spark) branch master updated (a5910a2dbb00 -> df563904ccb1)
gurwls223
(spark) branch master updated: [SPARK-47480][PYTHON][CONNECT][TESTS] Enable doctest for `createDataFrame`
gurwls223
(spark) branch branch-3.5 updated: [SPARK-47473][SQL] Fix correctness issue of converting postgres INFINITY timestamps
yao
(spark) branch master updated (85bf7615f85e -> ad8ac17dbdfa)
yao
(spark) branch branch-3.4 updated: [SPARK-47455][BUILD] Fix resource leak during the initialization of `scalaStyleOnCompileConfig` in `SparkBuild.scala`
yangjie01
(spark) branch branch-3.5 updated: [SPARK-47455][BUILD] Fix resource leak during the initialization of `scalaStyleOnCompileConfig` in `SparkBuild.scala`
yangjie01
(spark) branch master updated (c3a04fa59ce1 -> 85bf7615f85e)
yangjie01
(spark) branch master updated: [SPARK-47447][SQL] Allow reading Parquet TimestampLTZ as TimestampNTZ
gengliang
(spark) branch master updated: [SPARK-47007][SQL][PYTHON][R][CONNECT] Add the `map_sort` function
maxgekk
(spark) branch branch-3.4 updated: [SPARK-47472][INFRA][3.4] Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`
dongjoon
(spark) branch master updated (bc378f4ff5e2 -> 61d7b0f24fc9)
dongjoon
(spark) branch master updated: [SPARK-47330][SQL][TESTS] XML: Added XmlExpressionsSuite
gurwls223
(spark) branch master updated: [SPARK-47309][SQL] XML: Add schema inference tests for value tags
gurwls223
(spark) branch master updated: [SPARK-47468][BUILD] Exclude `logback` dependency from SBT like Maven
dongjoon
(spark) branch master updated: [SPARK-47454][PYTHON][CONNECT][TESTS][FOLLOWUP] Further split `pyspark.sql.tests.test_dataframe`
gurwls223
(spark) branch master updated: [SPARK-47449][SS] Refactor and split list/timer unit tests
kabhwan
(spark) branch master updated: [SPARK-47464][INFRA] Update `labeler.yml` for module `common/sketch` and `common/variant`
dongjoon
(spark) branch master updated (90560dce85b0 -> db531c6ee719)
dongjoon
(spark) branch master updated: [SPARK-47458][CORE] Fix the problem with calculating the maximum concurrent tasks for the barrier stage
tgraves
(spark) branch master updated (b6a836946311 -> a6bffcc3e5f0)
dongjoon
(spark) branch master updated (ef94f7094989 -> b6a836946311)
gurwls223
(spark) branch master updated: [SPARK-47452][INFRA] Use `Ubuntu 22.04` in `dev/infra/Dockerfile`
dongjoon
(spark) branch master updated (5f48931fcdf7 -> 5e42ecc8163a)
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47435][SPARK-45561][SQL][3.5] Fix overflow issue of MySQL UNSIGNED TINYINT caused by
yao
(spark) branch master updated (681b41f0808e -> 5f48931fcdf7)
dongjoon
(spark) branch master updated (e01ed0da22f2 -> 681b41f0808e)
wenchen
(spark) branch master updated (9f8147c2a8d2 -> e01ed0da22f2)
dongjoon
(spark) branch master updated (acf17fd67217 -> 9f8147c2a8d2)
kabhwan
(spark) branch master updated (cb20fcae951d -> acf17fd67217)
dongjoon
(spark) branch master updated (51e8634a5883 -> cb20fcae951d)
dongjoon
(spark) branch master updated: [SPARK-47380][CONNECT] Ensure on the server side that the SparkSession is the same
hvanhovell
(spark) branch master updated: [SPARK-47446][CORE] Make `BlockManager` warn before `removeBlockInternal`
dongjoon
(spark) branch master updated: [SPARK-47383][CORE] Support `spark.shutdown.timeout` config
dongjoon
(spark) branch master updated: [SPARK-47435][SQL] Fix overflow issue of MySQL UNSIGNED TINYINT caused by SPARK-45561
dongjoon
(spark) branch master updated (4dc362dbc6c0 -> 1aafe60b3e76)
dongjoon
(spark) branch master updated: [SPARK-47438][BUILD] Upgrade jackson to 2.17.0
dongjoon
(spark) branch master updated: [MINOR][DOCS] Add `Web UI` link to `Other Documents` section of index.md
dongjoon
(spark) branch branch-3.4 updated: [SPARK-47434][WEBUI] Fix `statistics` link in `StreamingQueryPage`
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47434][WEBUI] Fix `statistics` link in `StreamingQueryPage`
dongjoon
(spark) branch master updated (d3f12df6e09e -> 9b466d329c3c)
dongjoon
(spark) branch master updated: [SPARK-47437][PYTHON][CONNECT] Correct the error class for `DataFrame.sort*`
gurwls223
(spark) branch master updated (08866c280f87 -> 42c4dad62dcd)
gurwls223
(spark) branch master updated: [SPARK-47439][PYTHON] Document Python Data Source API in API reference page
gurwls223
(spark) branch master updated: [MINOR][TESTS] Collation - extending golden file coverage
maxgekk
(spark) branch master updated (310dd5294267 -> 3f171ce3f43b)
ruifengz
(spark) branch master updated: [SPARK-45891][DOCS] Update README with more details
wenchen
(spark) branch branch-3.4 updated (be0e44e59b3e -> b4e2c6750cb3)
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47432][PYTHON][CONNECT][DOCS][3.5] Add `pyarrow` upper bound requirement, `<13.0.0`
dongjoon
(spark) branch branch-3.4 updated: [SPARK-45141][PYTHON][INFRA][TESTS] Pin `pyarrow==12.0.1` in CI
dongjoon
(spark) branch branch-3.5 updated: [SPARK-45141][PYTHON][INFRA][TESTS] Pin `pyarrow==12.0.1` in CI
dongjoon
(spark) branch master updated: [SPARK-47426][BUILD] Upgrade Guava used by the connect module to `33.1.0-jre`
dongjoon
(spark-website) branch asf-site updated: Update the organization in committers.md (#509)
dongjoon
[PR] Update the organization in committers.md [spark-website]
via GitHub
Re: [PR] Update the organization in committers.md [spark-website]
via GitHub
Re: [PR] Update the organization in committers.md [spark-website]
via GitHub
(spark) branch master updated: [SPARK-47377][PYTHON][CONNECT][TESTS][FOLLOWUP] Factor out more tests from `SparkConnectSQLTestCase`
gurwls223
Earlier messages
Later messages