[ANNOUNCE] Apache CarbonData 1.5.2 release

2019-02-04 Thread Raghunandan S
Hi All, Apache CarbonData community is pleased to announce the release of the Version 1.5.2 in The Apache Software Foundation (ASF). CarbonData is a high-performance data solution that supports various data analytic scenarios, including BI analysis, ad-hoc SQL query, fast filter lookup on detail

Re: [Discussion]read latest schema in case of external table and file format

2019-02-04 Thread akashrn5
Hi rahul, Actually we are not skipping the old file, currently we are just listing the carbondata files in the location and then take first one to infer the schema, but now i just take the latest carbon data file to infer schema, and while giving the data, if the column is not present in correspon

Re: [Discussion]read latest schema in case of external table and file format

2019-02-04 Thread akashrn5
Hi Liang, When we create a table using location in file format case or when i create an external table from a location, user can place multiple carbondata files with different schema in that location and want to read the data at once, in that scenario we can expect the above condition. So curren

Re: [Discussion]read latest schema in case of external table and file format

2019-02-04 Thread Rahul Kumar
hi akash, I have one concern related to this change: *Concern*: why we are skipping old datafile? if user is not giving the schema then also i think we should read old data file. we can fill column *d* and *e* with *None* value. i guess *if data file is present at given location it means

Re: [Discussion]read latest schema in case of external table and file format

2019-02-04 Thread Liang Chen
Hi Can you explain which scenario will generate two carbondata files with different schema? Regards Liang akashrn5 wrote > Hi dev, > > Currently we have a validation that if there are two carbondata files in a > location with different schema, then we fail the query. I think there is > no > n

Re: [VOTE] Apache CarbonData 1.5.2(RC2) release

2019-02-04 Thread Raghunandan S
Hi all PMC vote has passed for Apache Carbondata 1.5.2 release, the result as below: +1(binding): 5(Jacky, Kumar Vishal, Ravindra, David CaiQiang, Liang Chen) +1(non-binding) : 3 Thanks all for your vote. Regards Raghunandan On Mon, Feb 4, 2019 at 3:50 PM Liang Chen wrote: > Hi > > +

Re: [VOTE] Apache CarbonData 1.5.2(RC2) release

2019-02-04 Thread Liang Chen
Hi +1 Regards Liang sraghunandan wrote > Hi > > > I submit the Apache CarbonData 1.5.2 (RC2) for your vote. > > > 1.Release Notes: > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220&version=12344321 > > > Some key features and improvements in this release:

[Discussion]read latest schema in case of external table and file format

2019-02-04 Thread Akash Nilugal
Hi dev, Currently we have a validation that if there are two carbondata files in a location with different schema, then we fail the query. I think there is no need to fail. If you see the parquet behavior also we cna understand. Here i think failing is not good, we can read the latets schema from