xorsum created SPARK-47829:
------------------------------
Summary: Text Datasource supports zstd compression codec
Key: SPARK-47829
URL: https://issues.apache.org/jira/browse/SPARK-47829
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.5.0
Reporter: xorsum
{code:java}
drop table if exists tmp_compressed_datasource;CREATE TABLE
tmp_compressed_datasource (key STRING) USING TEXT OPTIONS (compression
'zstd');INSERT INTO TABLE tmp_compressed_datasource values
('a'),('b'),('c');select * from tmp_compressed_datasource ;
{code}
{code:java}
java.lang.IllegalArgumentException: Codec [zstd] is not available. Known codecs
are bzip2, deflate, uncompressed, lz4, gzip, snappy, none.
at
org.apache.spark.sql.catalyst.util.CompressionCodecs$.getCodecClassName(CompressionCodecs.scala:53)
at
org.apache.spark.sql.execution.datasources.text.TextOptions.$anonfun$compressionCodec$1(TextOptions.scala:38)
at scala.Option.map(Option.scala:230)
at
org.apache.spark.sql.execution.datasources.text.TextOptions.<init>(TextOptions.scala:38)
at
org.apache.spark.sql.execution.datasources.text.TextOptions.<init>(TextOptions.scala:33)
at
org.apache.spark.sql.execution.datasources.text.TextFileFormat.prepareWrite(TextFileFormat.scala:72)
at
org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:123)
at
org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:190)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
