[ https://issues.apache.org/jira/browse/DRILL-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16935943#comment-16935943 ]
benj commented on DRILL-6958: ----------------------------- In the next example, with a table with a column that contain a piece of json like {code:sql} SELECT * FROM ....`example.parquet` LIMIT 2; +---------+------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | hash | date | info | +---------+------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | B29C56F | 2019-09-23 | {"Number": 322, "scans": {"nameofprocess": {"detection": false, "version": "1.2"}}, {"othername": {"detection": true, "version": "0.1"}}} | | C28956E | 2019-09-22 | {"Number": 312, "scans": {"thirdname": {"detection": false, "version": "1.0"}}} | +---------------+------------------+-------------------------------------------------------------------------------------------------------------------------------+ SELECT typeof(hash) AS hash, typeof(`date`) AS `date`, typeof(info) AS info FROM ....`example.parquet` LIMIT 1; +---------+--------+------+ | hash | date | info | +---------+--------+------+ | VARCHAR | DATE | MAP | +---------+--------+------+ {code} It's not possible to push in a right way into a CSV file because of the presence of separator and quote inside the json. And there is no possibility to manually avoid this problem with a change of separator or introduce quote because the type MAP is not convertible in VARCHAR (DRILL-7375), so it's not possible to manually concatenate data > CTAS csv with option > -------------------- > > Key: DRILL-6958 > URL: https://issues.apache.org/jira/browse/DRILL-6958 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Text & CSV > Affects Versions: 1.15.0, 1.16.0 > Reporter: benj > Priority: Major > > Currently, it may be difficult to produce well-formed CSV with CTAS (see > comment below). > It appears necessary to have some additional/configuratble options to write > CSV file with CTAS : > * possibility to change/define the separator, > * possibility to write or not the header, > * possibility to force the write of only 1 file instead of lot of parts, > * possibility to force quoting > * possibility to use/change escape char > * ... -- This message was sent by Atlassian Jira (v8.3.4#803005)