Hi Yun, Drill is designed to query large datasets. There is no specific limit on the size, it works well even when data is in hundreds of GBs.
DATA_READ ERROR has something to do with the data in your file. The data in some of the columns may not be consistent with the datatype. Please refer to this link for one such example - https://stackoverflow.com/questions/40217328/apache-drill-mysql-and-data-read-error-failure-while-attempting-to-read-from Thanks, Prasad On Thu, Nov 2, 2017 at 7:46 AM, Yun Liu <y....@castsoftware.com> wrote: > Hi, > > I've been using Apache Drill actively and just wondering what is the > capacity of Drill? I have a json file which is 390MB and it keeps throwing > me an DATA_READ ERROR. I have another json file with exact same format but > only 150MB and it's processing fine. When I did a *select* on the large > json, it returns successfully for some of the fields. None of these errors > really apply to me. So I am trying to understand the capacity of the json > files Drill supports up to. Or if there's something else I missed. > > Thanks, > > Yun Liu > Solutions Delivery Consultant > 321 West 44th St | Suite 501 | New York, NY 10036 > +1 212.871.8355 office | +1 646.752.4933 mobile > > CAST, Leader in Software Analysis and Measurement > Achieve Insight. Deliver Excellence. > Join the discussion http://blog.castsoftware.com/ > LinkedIn<http://www.linkedin.com/companies/162909> | Twitter< > http://twitter.com/onquality> | Facebook<http://www.facebook. > com/pages/CAST/105668942817177> > >
