devinjdangelo commented on issue #7867:
URL:
https://github.com/apache/arrow-datafusion/issues/7867#issuecomment-1770912986
You can specify writer_version and in fact _almost_ all options without
quotes (see below). There are some special cases where datafusion's options
parser relies on special characters that `sqlparser-rs` will complain about if
it is not wrapped in single quotes. Notably compression options which contain
'(' and ')' as well as column specific options using '::'. We could potentially
make the UX in those cases nicer by modifying sqlparser-rs to accept those
special syntax.
I think cleaning up the error messages and documentation to make all of this
more clear is a good idea!
```sql
# Copy parquet with all supported statment overrides
query IT
COPY source_table
TO 'test_files/scratch/copy/table_with_options'
(format parquet,
single_file_output false,
compression snappy,
'compression::col1' 'zstd(5)',
'compression::col2' snappy,
max_row_group_size 12345,
data_pagesize_limit 1234,
write_batch_size 1234,
writer_version 2.0,
dictionary_page_size_limit 123,
created_by 'DF copy.slt',
column_index_truncate_length 123,
data_page_row_count_limit 1234,
bloom_filter_enabled true,
'bloom_filter_enabled::col1' false,
'bloom_filter_fpp::col2' 0.456,
'bloom_filter_ndv::col2' 456,
encoding plain,
'encoding::col1' DELTA_BINARY_PACKED,
'dictionary_enabled::col2' true,
dictionary_enabled false,
statistics_enabled page,
'statistics_enabled::col2' none,
max_statistics_size 123,
bloom_filter_fpp 0.001,
bloom_filter_ndv 100
)
----
2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
