Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

beliefer commented on code in PR #44786: URL: https://github.com/apache/spark/pull/44786#discussion_r1460278740 ## connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala: ## @@ -110,10 +110,12 @@ private[sql] object AvroUtils extends Logging { case co

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

beliefer commented on code in PR #44786: URL: https://github.com/apache/spark/pull/44786#discussion_r1460278740 ## connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala: ## @@ -110,10 +110,12 @@ private[sql] object AvroUtils extends Logging { case co

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

dongjoon-hyun commented on code in PR #44786: URL: https://github.com/apache/spark/pull/44786#discussion_r1459618369 ## connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala: ## @@ -110,10 +110,12 @@ private[sql] object AvroUtils extends Logging { ca

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

beliefer commented on code in PR #44786: URL: https://github.com/apache/spark/pull/44786#discussion_r1458650779 ## connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala: ## @@ -110,10 +110,12 @@ private[sql] object AvroUtils extends Logging { case co

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

yaooqinn commented on PR #44786: URL: https://github.com/apache/spark/pull/44786#issuecomment-1899516986 Thank you very much @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

dongjoon-hyun commented on PR #44786: URL: https://github.com/apache/spark/pull/44786#issuecomment-1898811626 Merged to master for Apache Spark 4.0.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

dongjoon-hyun closed pull request #44786: [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files URL: https://github.com/apache/spark/pull/44786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

dongjoon-hyun commented on code in PR #44786: URL: https://github.com/apache/spark/pull/44786#discussion_r1457695414 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -3625,7 +3625,23 @@ object SQLConf { .version("2.4.0") .intConf .

Re: [PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

dongjoon-hyun commented on code in PR #44786: URL: https://github.com/apache/spark/pull/44786#discussion_r1457693863 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -3625,7 +3625,23 @@ object SQLConf { .version("2.4.0") .intConf .

[PR] [SPARK-46759][SQL][AVRO] Codec xz and zstandard support compression level for avro files [spark]

yaooqinn opened a new pull request, #44786: URL: https://github.com/apache/spark/pull/44786 ### What changes were proposed in this pull request? This PR introduces 2 keys in the form of 'spark.sql.avro.$codecName.level' just like the existing 'spark.sql.avro.deflate.le