File size limit for CTAS?

2016-01-21 Thread Matt
Converting CSV files to Parquet with CTAS, and getting errors on some larger files: With a source file of 16.34GB (as reported in the HDFS explorer): ~~~ create table `/parquet/customer_20151017` partition by (date_tm) AS select * from `/csv/customer/customer_20151017.csv`; Error: SYSTEM

Re: File size limit for CTAS?

2016-01-21 Thread rahul challapalli
Ignoring the CTAS part can you try running the select query and see if it completes. My suspicion is that some record/field in your large file is causing drill to break. Also it would be helpful if you can give more information from the drillbit.log when this error happens (Search for