commits
Thread
Date
Earlier messages
Later messages
Messages by Thread
(spark) branch master updated: [SPARK-46812][CONNECT][FOLLOW-UP] Make `handleCreateResourceProfileCommand` private
dongjoon
(spark) branch master updated: [SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework
gengliang
(spark) branch master updated (3da52fb4490e -> 75d43dd05757)
gengliang
(spark) branch master updated: [SPARK-47798][SQL] Enrich the error message for the reading failures of decimal values
yao
(spark) branch branch-3.4 updated: [MINOR][DOCS] Make the link of spark properties with YARN more accurate
dongjoon
(spark) branch branch-3.5 updated: [MINOR][DOCS] Make the link of spark properties with YARN more accurate
dongjoon
(spark) branch master updated: [MINOR][DOCS] Make the link of spark properties with YARN more accurate
dongjoon
(spark) branch master updated: [SPARK-47001][SQL] Pushdown verification in optimizer
wenchen
(spark) branch branch-3.4 updated: [MINOR][DOCS] Clarify relation between grouping API and `spark.sql.execution.arrow.maxRecordsPerBatch`
gurwls223
(spark) branch branch-3.5 updated: [MINOR][DOCS] Clarify relation between grouping API and `spark.sql.execution.arrow.maxRecordsPerBatch`
gurwls223
(spark) branch master updated: [MINOR][DOCS] Clarify relation between grouping API and `spark.sql.execution.arrow.maxRecordsPerBatch`
gurwls223
(spark) branch branch-3.5 updated: [SPARK-47704][SQL] JSON parsing fails with "java.lang.ClassCastException" when spark.sql.json.enablePartialResults is enabled
gurwls223
(spark) branch master updated: [SPARK-47704][SQL] JSON parsing fails with "java.lang.ClassCastException" when spark.sql.json.enablePartialResults is enabled
gurwls223
(spark) branch master updated: [SPARK-47274][PYTHON][SQL] Provide more useful context for PySpark DataFrame API errors
wenchen
(spark) branch master updated (8da9a70f701a -> b637bb92a2f8)
wenchen
(spark) branch master updated: [SPARK-47725][INFRA][FOLLOW-UP] Do not run scheduled job in forked repository
gurwls223
(spark) branch master updated: [MINOR][PYTHON][TESTS] Enable `test_udf_cache` parity test
ruifengz
(spark) branch master updated: [SPARK-47366][SQL][PYTHON] Add VariantVal for PySpark
gurwls223
(spark) branch master updated: [SPARK-41811][PYTHON][CONNECT] Implement `SQLStringFormatter` with `WithRelations`
gurwls223
(spark-connect-go) branch master updated: [MINOR] Make readme easier to follow
gurwls223
(spark) branch master updated: [SPARK-47595][STREAMING] Streaming: Migrate logError with variables to structured logging framework
gengliang
(spark) branch master updated: [SPARK-47781][SPARK-47791][SPARK-47798][DOCS][FOLLOWUP] Update the decimal mapping remarks with JDBC data sources
dongjoon
(spark) branch master updated: [SPARK-47693][SQL] Add optimization for lowercase comparison of UTF8String used in UTF8_BINARY_LCASE collation
wenchen
(spark) branch master updated: [SPARK-47775][SQL] Support remaining scalar types in the variant spec
wenchen
(spark) branch master updated: [SPARK-47680][SQL] Add variant_explode expression
wenchen
(spark) branch master updated (0d7c07047a62 -> 3082319bb1d5)
yao
(spark) branch branch-3.5 updated: [SPARK-47790][BUILD][3.5] Upgrade `commons-io` to 2.16.1
dongjoon
(spark) branch master updated: [SPARK-47790][BUILD] Upgrade `commons-io` to 2.16.1
dongjoon
(spark) branch master updated: [SPARK-47706][BUILD] Bump json4s 4.0.7
dongjoon
(spark) branch master updated (520f3b1c192b -> b53ec00421b1)
yao
(spark) branch master updated: [SPARK-47593][CORE] Connector module: Migrate logWarn with variables to structured logging framework
gengliang
(spark) branch master updated: [SPARK-47776][SS] Disallow binary inequality collation be used in key schema of stateful operator
kabhwan
(spark) branch branch-3.5 updated: [SPARK-47083][BUILD] Upgrade `commons-codec` to 1.16.1
dongjoon
(spark) branch master updated: [SPARK-47761][SQL] Oracle: Support reading AnsiIntervalTypes
yao
(spark) branch branch-3.5 updated: [SPARK-47182][BUILD] Exclude `commons-(io|lang3)` transitive dependencies from `commons-compress` and `avro*`
dongjoon
(spark) branch master updated: [SPARK-47786] SELECT DISTINCT (*) should not become SELECT DISTINCT struct(*) (revert to previous behavior)
wenchen
(spark) branch master updated: [SPARK-47586][SQL] Hive module: Migrate logError with variables to structured logging framework
gengliang
(spark) branch master updated (0eb58a06a7d4 -> d1e1a1907412)
dongjoon
(spark) branch master updated: [SPARK-47785][BUILD][TESTS] Upgrade `bouncycastle` to 1.78
dongjoon
(spark) branch master updated (07b1346db477 -> a68337892246)
gengliang
(spark) branch master updated (2793397140af -> 07b1346db477)
gengliang
(spark) branch master updated (1586652b6421 -> 2793397140af)
dongjoon
(spark) branch master updated: [MINOR][PS][TESTS] Make `test_compare` deterministic
dongjoon
(spark) branch master updated: [SPARK-47779][PS][TESTS] Add a helper function to sort PS Frame/Series
ruifengz
(spark) branch master updated: [SPARK-47081][CONNECT][FOLLOW] Improving the usability of the Progress Handler
gurwls223
(spark) branch branch-3.4 updated: [SPARK-47774][INFRA][3.4] Remove redundant rules from `MimaExcludes`
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47774][INFRA][3.5] Remove redundant rules from `MimaExcludes`
dongjoon
(spark) branch master updated: [SPARK-47774][INFRA] Remove redundant rules from `MimaExcludes`
dongjoon
(spark) branch master updated (149ac0f8893b -> 83f3a34df7a0)
ruifengz
(spark) branch master updated: [SPARK-47581][CORE] SQL catalyst: Migrate logWarning with variables to structured logging framework
gengliang
(spark) branch master updated: [SPARK-47754][SQL] Postgres: Support reading multidimensional arrays
dongjoon
(spark) branch master updated (319edfdc5cd6 -> edba7b9f9119)
ruifengz
(spark) branch branch-3.4 updated: [SPARK-47770][INFRA] Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return `false` instead of failing
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47770][INFRA] Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return `false` instead of failing
dongjoon
(spark) branch master updated (08c49637795f -> 319edfdc5cd6)
wenchen
(spark) branch master updated: [SPARK-47770][INFRA] Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return `false` instead of failing
yangjie01
(spark) branch master updated: [MINOR][TESTS] Make `check_invalid_args` reusable in parity test
gurwls223
(spark) branch master updated: [SPARK-47589][SQL] Hive-Thriftserver: Migrate logError with variables to structured logging framework
gengliang
(spark) branch master updated: [SPARK-47737][PYTHON] Bump PyArrow to 10.0.0
dongjoon
(spark) branch master updated: [SPARK-47725][INFRA] Set up the CI for pyspark-connect package
dongjoon
(spark) branch master updated: [SPARK-47504][SQL] Resolve AbstractDataType simpleStrings for StringTypeCollated
wenchen
(spark) branch master updated: [SPARK-47681][SQL] Add schema_of_variant expression
wenchen
(spark) branch master updated: [SPARK-47746] Implement ordinal-based range encoding in the RocksDBStateEncoder
kabhwan
(spark) branch master updated: [SPARK-47748][BUILD] Upgrade `zstd-jni` to 1.5.6-2
yao
(spark) branch branch-3.5 updated: [SPARK-47762][PYTHON][CONNECT] Add pyspark.sql.connect.protobuf into setup.py
gurwls223
(spark) branch master updated: [SPARK-47762][PYTHON][CONNECT] Add pyspark.sql.connect.protobuf into setup.py
gurwls223
(spark) branch master updated (f0d8f82f8c45 -> 211afd4ab526)
ruifengz
(spark) branch master updated (eb8e99721714 -> f0d8f82f8c45)
yao
(spark) branch master updated: [SPARK-47657][SQL] Implement collation filter push down support per file source
wenchen
(spark) branch master updated: [SPARK-47713][SQL][CONNECT] Fix a self-join failure
wenchen
(spark) branch master updated: [SPARK-47558][SS] State TTL support for ValueState
kabhwan
(spark) branch master updated (ad2367c55aeb -> f576b8542f2e)
wenchen
(spark) branch master updated: [MINOR][PYTHON][SS][TESTS] Drop the tables after being used at `test_streaming_foreach_batch`
gurwls223
(spark) branch master updated (b299b2bc06a9 -> cc6c0eb1bee6)
gurwls223
(spark) branch branch-3.5 updated: [SPARK-47299][PYTHON][DOCS] Use the same `versions.json` in the dropdown of different versions of PySpark documents
kabhwan
(spark) branch master updated (0c992b205946 -> b299b2bc06a9)
kabhwan
(spark) branch master updated (e92e8f5441a7 -> 0c992b205946)
gurwls223
(spark) branch branch-3.4 updated: [SPARK-47734][PYTHON][TESTS][3.4] Fix flaky DataFrame.writeStream doctest by stopping streaming query
kabhwan
(spark) branch master updated: [SPARK-47744] Add support for negative-valued bytes in range encoder
kabhwan
(spark) branch master updated (d7430124191a -> f7dff4aa0c8f)
gurwls223
(spark) branch master updated: [SPARK-47753][PYTHON][CONNECT][TESTS] Make pyspark.testing compatible with pyspark-connect
gurwls223
(spark) branch master updated: [SPARK-47751][PYTHON][CONNECT] Make pyspark.worker_utils compatible with pyspark-connect
gurwls223
(spark) branch master updated (644687b66e1a -> 4d9dbb35aacb)
gurwls223
(spark) branch master updated: [SPARK-47709][BUILD] Upgrade tink to 1.13.0
dongjoon
(spark) branch master updated: [SPARK-47727][PYTHON] Make SparkConf to root level to for both SparkSession and SparkContext
dongjoon
(spark) branch master updated: [SPARK-47738][BUILD] Upgrade Kafka to 3.7.0
dongjoon
(spark) branch master updated (42dc815b8446 -> 7385f19c7539)
gengliang
(spark) branch master updated (11abc64a731d -> 42dc815b8446)
gengliang
(spark) branch master updated: [SPARK-47094][SQL] SPJ : Dynamically rebalance number of buckets when they are not equal
sunchao
(spark) branch master updated (1efbf43160aa -> d1ace24f8fac)
gengliang
(spark) branch branch-3.4 updated: [SPARK-45445][BUILD][3.4] Upgrade snappy to 1.1.10.5
dongjoon
(spark) branch branch-3.5 updated: [SPARK-45445][BUILD][3.5] Upgrade snappy to 1.1.10.5
dongjoon
(spark) branch master updated: [SPARK-47310][SS] Add micro-benchmark for merge operations for multiple values in value portion of state store
kabhwan
(spark) branch master updated: [SPARK-47577][CORE][PART2] Migrate logError with variables to structured logging framework
gengliang
(spark) branch branch-3.4 updated: [SPARK-47111][SQL][TESTS][3.4] Upgrade `PostgreSQL` JDBC driver to 42.7.2 and docker image to 16.2
dongjoon
(spark) branch branch-3.4 updated: [SPARK-46411][BUILD][3.4] Change to use `bcprov/bcpkix-jdk18on` for UT
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47111][SQL][TESTS][3.5] Upgrade `PostgreSQL` JDBC driver to 42.7.2 and docker image to 16.2
dongjoon
(spark) branch master updated: [SPARK-47719][SQL] Change spark.sql.legacy.timeParserPolicy default to CORRECTED
gengliang
(spark) branch branch-3.5 updated: [SPARK-46411][BUILD][3.5][FOLLOWUP] Fix pom.xml file in common/network-common module
dongjoon
(spark) branch branch-3.5 updated: [SPARK-46411][BUILD] Change to use `bcprov/bcpkix-jdk18on` for UT
dongjoon
(spark) branch branch-3.4 updated: [SPARK-44441][BUILD] Upgrade `bcprov-jdk15on` and `bcpkix-jdk15on` to 1.70
dongjoon
(spark) branch branch-3.4 updated: [SPARK-44393][BUILD] Upgrade `H2` from 2.1.214 to 2.2.220
dongjoon
(spark) branch master updated: [SPARK-47511][SQL][FOLLOWUP] Rename the config REPLACE_NULLIF_USING_WITH_EXPR to be more general
dongjoon
(spark) branch master updated (97e63ff035b5 -> 12d0367cd304)
dongjoon
(spark) branch master updated (aeb082e06091 -> 97e63ff035b5)
dongjoon
(spark) branch master updated (d5620cbe9a2e -> aeb082e06091)
gurwls223
(spark) branch master updated (0107435cb39d -> d5620cbe9a2e)
wenchen
(spark) branch branch-3.5 updated: [SPARK-47734][PYTHON][TESTS] Fix flaky DataFrame.writeStream doctest by stopping streaming query
gurwls223
(spark) branch master updated (b9ca91dde94c -> 0107435cb39d)
gurwls223
(spark) branch branch-3.5 updated: [SPARK-47568][SS][3.5] Fix race condition between maintenance thread and load/commit for snapshot files
gurwls223
(spark) branch master updated: [SPARK-47712][CONNECT] Allow connect plugins to create and process Datasets
gurwls223
(spark) branch master updated: [SPARK-47081][CONNECT][FOLLOW-UP] Add the `shell` module into PyPI package
gurwls223
(spark) branch master updated: [SPARK-47723][CORE][TESTS] Introduce a tool that can sort alphabetically enumeration field in `LogEntry` automatically
gengliang
(spark-website) branch asf-site updated: Update spark connect page based on pr feedback (#512)
dongjoon
(spark) branch master updated: [SPARK-46812][PYTHON][TESTS][FOLLOWUP] Check should_test_connect and pyarrow to skip tests
dongjoon
(spark) branch master updated: [SPARK-47598][CORE] MLLib: Migrate logError with variables to structured logging framework
gengliang
(spark) branch master updated: [SPARK-47610][CONNECT][FOLLOWUP] Add -Dio.netty.tryReflectionSetAccessible=true for spark-connect-scala-client
dongjoon
(spark) branch master updated (5ca3467b8653 -> 25fc67fa114d)
dongjoon
(spark) branch master updated: [SPARK-47729][PYTHON][TESTS] Get the proper default port for pyspark-connect testcases
dongjoon
(spark) branch master updated: [SPARK-47689][SQL][FOLLOWUP] More accurate file path in TASK_WRITE_FAILED error
wenchen
(spark) branch master updated: Revert "[SPARK-47708][CONNECT] Do not log gRPC exception to stderr in PySpark
wenchen
(spark) branch master updated (f6999df0c7f0 -> bffb02d14fa4)
gurwls223
(spark) branch master updated: [SPARK-47081][CONNECT] Support Query Execution Progress
wenchen
(spark) branch master updated (d75c77562d27 -> 3f6ac60e9966)
gengliang
(spark) branch master updated (d272a1b4367e -> d75c77562d27)
dongjoon
(spark) branch master updated: [SPARK-47724][PYTHON][TESTS] Add an environment variable for testing remote pure Python library
gurwls223
(spark) branch master updated: [SPARK-47705][INFRA][FOLLOWUP] Sort LogKey alphabetically and build a test to ensure it
gengliang
(spark) branch master updated (447f8aff6c26 -> 678aeb7ef708)
gurwls223
(spark) branch master updated (d87ac8ef49db -> 447f8aff6c26)
dongjoon
(spark) branch master updated: [SPARK-47708][CONNECT] Do not log gRPC exception to stderr in PySpark
gurwls223
(spark) branch master updated: [SPARK-47210][SQL] Addition of implicit casting without indeterminate support
wenchen
(spark) branch master updated: [SPARK-47722][SS] Wait until RocksDB background work finish before closing
gurwls223
(spark) branch master updated: [SPARK-47721][DOC] Guidelines for the Structured Logging Framework
gurwls223
(spark) branch master updated (7dec5eb14644 -> 6a0555c39b2d)
gurwls223
(spark) branch master updated: [SPARK-47705][INFRA] Sort LogKey alphabetically and build a test to ensure it
gengliang
(spark) branch master updated (a427a4586177 -> 1515c567e4f0)
kabhwan
(spark) branch master updated: [SPARK-47710][SQL][DOCS] Postgres: Document Mapping Spark SQL Data Types from PostgreSQL
dongjoon
(spark) branch master updated (1404d1f801c7 -> efe437ef9f9e)
dongjoon
(spark) branch master updated: [SPARK-47715][BUILD][STS] Upgrade hive-service-rpc 4.0.0
dongjoon
[PR] update spark connect page based on pr feedback [spark-website]
via GitHub
Re: [PR] update spark connect page based on pr feedback [spark-website]
via GitHub
Re: [PR] update spark connect page based on pr feedback [spark-website]
via GitHub
Re: [PR] update spark connect page based on pr feedback [spark-website]
via GitHub
Re: [PR] update spark connect page based on pr feedback [spark-website]
via GitHub
Re: [PR] update spark connect page based on pr feedback [spark-website]
via GitHub
(spark) branch master updated (d16183d044a6 -> d487acd3008b)
yao
(spark) branch master updated: [SPARK-47653][SS] Add support for negative numeric types and range scan key encoder
kabhwan
(spark) branch master updated (62f90ec6d32f -> 6e3a42515067)
gurwls223
(spark) branch master updated: [SPARK-47452][INFRA][FOLLOWUP] Enforce to install `six` to `Python 3.10`
dongjoon
(spark) branch master updated: [SPARK-47701][SQL][TESTS] Postgres: Add test for Composite and Range types
dongjoon
(spark) branch master updated: [SPARK-45733][PYTHON][TESTS][FOLLOWUP] Skip `pyspark.sql.tests.connect.client.test_client` if not should_test_connect
dongjoon
(spark) branch master updated (2c2a2adc3275 -> 344f640b2f35)
dongjoon
(spark) branch master updated (49b7b6b9fe6b -> 2c2a2adc3275)
kabhwan
(spark) branch master updated (55b5ff6f45fd -> 49b7b6b9fe6b)
dongjoon
(spark) branch master updated: [SPARK-47669][SQL][CONNECT][PYTHON] Add `Column.try_cast`
ruifengz
(spark) branch master updated: [SPARK-47697][INFRA] Add Scala style check for invalid MDC usage
dongjoon
[PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
Re: [PR] Add Spark Connect page [spark-website]
via GitHub
(spark) branch master updated: [SPARK-47699][BUILD] Upgrade `gcs-connector` to 2.2.21 and add a note for 3.0.0
dongjoon
(spark) branch master updated (db0975cb2a1c -> e6144e4d75b7)
dongjoon
(spark) branch master updated (8d4e9647c971 -> db0975cb2a1c)
gengliang
(spark) branch master updated (ba98b7ae53f8 -> 8d4e9647c971)
yao
(spark) branch branch-3.4 updated: [SPARK-47666][SQL][3.4] Fix NPE when reading mysql bit array as LongType
yao
(spark) branch master updated: [SPARK-47689][SQL] Do not wrap query execution error during data writing
wenchen
(spark) branch master updated (03f4e45cd7e9 -> b1b1fde42bd8)
yao
(spark) branch branch-3.5 updated: [SPARK-47666][SQL][3.5] Fix NPE when reading mysql bit array as LongType
yao
(spark) branch master updated (a598f654066d -> 03f4e45cd7e9)
yangjie01
(spark) branch master updated: [SPARK-47664][PYTHON][CONNECT][TESTS][FOLLOW-UP] Add more tests
ruifengz
(spark) branch master updated (e2e6e09a24db -> 22771a6a87e7)
yao
(spark) branch master updated: [SPARK-47634][SQL] Add legacy support for disabling map key normalization
wenchen
(spark) branch master updated: [SPARK-47663][CORE][TESTS] add end to end test for task limiting according to different cpu and gpu configurations
weichenxu123
(spark) branch master updated: [SPARK-47686][SQL][TESTS] Use `=!=` instead of `!==` in `JoinHintSuite`
dongjoon
(spark) branch master updated (e9833495e13a -> c3da26051d0c)
yao
(spark) branch master updated: [SPARK-47679][SQL] Use `HiveConf.getConfVars` or Hive conf names directly
dongjoon
(spark) branch master updated (43cedc0b3dea -> 1fd308951668)
wenchen
(spark) branch master updated: [SPARK-46812][CONNECT][PYTHON] Make mapInPandas / mapInArrow support ResourceProfile
gurwls223
(spark) branch master updated (128f74b055d3 -> 58f44e8a00f3)
gurwls223
(spark) branch branch-3.4 updated: [SPARK-47676][BUILD] Clean up the removed `VersionsSuite` references
dongjoon
(spark) branch master updated: [SPARK-47676][BUILD] Clean up the removed `VersionsSuite` references
dongjoon
(spark) branch branch-3.5 updated: [SPARK-47676][BUILD] Clean up the removed `VersionsSuite` references
dongjoon
(spark) branch master updated (e86c499b008c -> 72fe58d716d4)
dongjoon
(spark) branch master updated: [SPARK-47674][CORE] Enable `spark.metrics.appStatusSource.enabled` by default
dongjoon
(spark) branch master updated: [SPARK-47659][CORE][TESTS] Improve `*LoggingSuite*`
gengliang
(spark) branch master updated: [SPARK-46840][SQL][TESTS] Add `CollationBenchmark`
wenchen
(spark) branch master updated (968cba200a5a -> 72c619e4f3ba)
yao
(spark) branch master updated: [SPARK-47664][PYTHON][CONNECT] Validate the column name with cached schema
gurwls223
(spark) branch master updated: [SPARK-47569][SQL] Disallow comparing variant
wenchen
(spark) branch branch-3.5 updated: [SPARK-45593][BUILD][3.5] Correct relocation connect guava dependency
yangjie01
(spark) branch master updated (a5ca5867298f -> 535734e3901f)
wenchen
Earlier messages
Later messages