[ https://issues.apache.org/jira/browse/SPARK-47829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated SPARK-47829: ----------------------------------- Labels: pull-request-available (was: ) > Text Datasource supports zstd compression codec > ----------------------------------------------- > > Key: SPARK-47829 > URL: https://issues.apache.org/jira/browse/SPARK-47829 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.5.0 > Reporter: xorsum > Priority: Minor > Labels: pull-request-available > Original Estimate: 24h > Remaining Estimate: 24h > > {code:java} > drop table if exists tmp_compressed_datasource;CREATE TABLE > tmp_compressed_datasource (key STRING) USING TEXT OPTIONS (compression > 'zstd');INSERT INTO TABLE tmp_compressed_datasource values > ('a'),('b'),('c');select * from tmp_compressed_datasource ; > {code} > {code:java} > java.lang.IllegalArgumentException: Codec [zstd] is not available. Known > codecs are bzip2, deflate, uncompressed, lz4, gzip, snappy, none. > at > org.apache.spark.sql.catalyst.util.CompressionCodecs$.getCodecClassName(CompressionCodecs.scala:53) > at > org.apache.spark.sql.execution.datasources.text.TextOptions.$anonfun$compressionCodec$1(TextOptions.scala:38) > at scala.Option.map(Option.scala:230) > at > org.apache.spark.sql.execution.datasources.text.TextOptions.<init>(TextOptions.scala:38) > at > org.apache.spark.sql.execution.datasources.text.TextOptions.<init>(TextOptions.scala:33) > at > org.apache.spark.sql.execution.datasources.text.TextFileFormat.prepareWrite(TextFileFormat.scala:72) > at > org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:123) > at > org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:190) > {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org